2024-09-10T18:01:31.384 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T18:01:31.400 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T18:01:31.486 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900198 branch: wip-guits-quincy-2024-09-09-1810-quincy description: orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} email: gabrioux@ibm.com first_in_suite: false job_id: '7900198' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-09-1810-quincy ceph: conf: global: mon election default strategy: 3 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(PG_ - \(OSD_ - \(OBJECT_ - \(POOL_APP_NOT_ENABLED\) mon_bind_msgr2: false sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: extra_system_packages: deb: - python3-pytest rpm: - python3-pytest flavor: default sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: quincy-orch-centos-9-teuth sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4625 sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: quincy-orch-centos-9-teuth suite_path: /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa suite_relpath: qa suite_repo: https://github.com/adk3798/ceph.git suite_sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d targets: smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEOpqYfBHVoDcJO41/zlRhQOVvryWySoK3RA5FL+/eKOWvZYpK1r3WfMOcR6joAJWNh5BLKod5izrAeOWWUmPU= smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHBJ+JB8Xn5UraSDvFz6pVECtEMlgdVEGvgBfaKeYJ+RuA3NLQoN5g9UoNegoveTvH1uoxffNmunZPaPS5OfR7A= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test_python.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_17:53:37 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T18:01:31.486 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa; will attempt to use it 2024-09-10T18:01:31.487 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa/tasks 2024-09-10T18:01:31.487 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T18:01:31.489 INFO:teuthology.task.internal:Checking packages... 2024-09-10T18:01:31.510 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '5be57395ea235535910512ae73ba8e7ff25fe588' 2024-09-10T18:01:31.510 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T18:01:31.510 INFO:teuthology.packaging:ref: None 2024-09-10T18:01:31.510 INFO:teuthology.packaging:tag: None 2024-09-10T18:01:31.510 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:01:31.510 INFO:teuthology.packaging:sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:01:31.511 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:01:31.647 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1606.g5be57395 2024-09-10T18:01:31.648 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T18:01:31.655 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T18:01:31.655 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T18:01:31.756 INFO:teuthology.task.internal:Saving configuration 2024-09-10T18:01:31.767 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T18:01:31.774 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T18:01:31.793 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900198', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 17:56:23.002757', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEOpqYfBHVoDcJO41/zlRhQOVvryWySoK3RA5FL+/eKOWvZYpK1r3WfMOcR6joAJWNh5BLKod5izrAeOWWUmPU='} 2024-09-10T18:01:31.816 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900198', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 17:56:23.000849', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHBJ+JB8Xn5UraSDvFz6pVECtEMlgdVEGvgBfaKeYJ+RuA3NLQoN5g9UoNegoveTvH1uoxffNmunZPaPS5OfR7A='} 2024-09-10T18:01:31.816 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T18:01:31.824 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-10T18:01:31.824 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-10T18:01:31.824 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T18:01:31.873 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd2b6babac0>, signals=[15]) 2024-09-10T18:01:31.873 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T18:01:31.879 INFO:teuthology.task.internal:Opening connections... 2024-09-10T18:01:31.880 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T18:01:31.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:01:31.956 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T18:01:31.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:01:32.033 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T18:01:32.039 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-09-10T18:01:32.057 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-09-10T18:01:32.058 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-09-10T18:01:32.115 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2024-09-10T18:01:32.115 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9" 2024-09-10T18:01:32.115 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2024-09-10T18:01:32.115 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:01:32.115 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9" 2024-09-10T18:01:32.116 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:01:32.116 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:01:32.116 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2024-09-10T18:01:32.116 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:01:32.116 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:01:32.116 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:01:32.116 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:01:32.116 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:01:32.116 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:01:32.117 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-09-10T18:01:32.137 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-09-10T18:01:32.154 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-09-10T18:01:32.154 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-09-10T18:01:32.211 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2024-09-10T18:01:32.211 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9" 2024-09-10T18:01:32.211 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2024-09-10T18:01:32.211 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:01:32.211 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9" 2024-09-10T18:01:32.211 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:01:32.211 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:01:32.211 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2024-09-10T18:01:32.211 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:01:32.211 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:01:32.212 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:01:32.212 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:01:32.212 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:01:32.212 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:01:32.212 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-09-10T18:01:32.245 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T18:01:32.254 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T18:01:32.307 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T18:01:32.307 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:01:32.310 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:01:32.326 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T18:01:32.348 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T18:01:32.348 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:01:32.367 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:01:32.400 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T18:01:32.444 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T18:01:32.459 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T18:01:32.459 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T18:01:32.459 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:01:32.460 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:01:32.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:01:32.475 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-09-10T18:01:32.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:01:32.477 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-09-10T18:01:32.531 INFO:teuthology.orchestra.run.smithi133.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:01:32.531 INFO:teuthology.task.kernel:Running kernel on smithi133: 5.14.0-503.el9.x86_64 2024-09-10T18:01:32.532 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum install -y kernel 2024-09-10T18:01:32.534 INFO:teuthology.orchestra.run.smithi099.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:01:32.535 INFO:teuthology.task.kernel:Running kernel on smithi099: 5.14.0-503.el9.x86_64 2024-09-10T18:01:32.535 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum install -y kernel 2024-09-10T18:01:37.227 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.2 MB 00:02 2024-09-10T18:01:38.928 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-09-10T18:01:40.182 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 1.4 MB/s | 8.2 MB 00:05 2024-09-10T18:01:42.049 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-10T18:01:43.578 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 6.5 MB 00:00 2024-09-10T18:01:45.496 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 19 kB 00:00 2024-09-10T18:01:46.267 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-10T18:01:46.400 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-10T18:01:48.125 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 19 kB 00:00 2024-09-10T18:01:48.871 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-10T18:01:51.730 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-10T18:01:53.761 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:01:53.761 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:01:53.839 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-10T18:01:53.845 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-10T18:01:53.845 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2024-09-10T18:01:53.845 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-10T18:01:53.845 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-10T18:01:53.845 INFO:teuthology.orchestra.run.smithi099.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:01:53.845 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-10T18:01:53.845 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:01:53.845 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:01:53.845 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:01:53.846 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:01:53.846 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-10T18:01:53.846 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-10T18:01:53.846 INFO:teuthology.orchestra.run.smithi099.stdout:Install 4 Packages 2024-09-10T18:01:53.846 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:01:53.846 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 79 M 2024-09-10T18:01:53.846 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 126 M 2024-09-10T18:01:53.847 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-10T18:01:54.031 INFO:teuthology.orchestra.run.smithi099.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 163 kB/s | 23 kB 00:00 2024-09-10T18:01:54.219 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T18:01:54.432 INFO:teuthology.orchestra.run.smithi099.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-10T18:01:54.840 INFO:teuthology.orchestra.run.smithi099.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 35 MB/s | 29 MB 00:00 2024-09-10T18:01:55.299 INFO:teuthology.orchestra.run.smithi099.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 25 MB/s | 35 MB 00:01 2024-09-10T18:01:55.299 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:01:55.299 INFO:teuthology.orchestra.run.smithi099.stdout:Total 54 MB/s | 79 MB 00:01 2024-09-10T18:01:55.979 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-10T18:01:56.266 INFO:teuthology.orchestra.run.smithi133.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:01:56.267 INFO:teuthology.orchestra.run.smithi133.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:01:56.343 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-10T18:01:56.343 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-10T18:01:56.349 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-09-10T18:01:56.359 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-09-10T18:01:56.359 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2024-09-10T18:01:56.359 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-09-10T18:01:56.359 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-09-10T18:01:56.359 INFO:teuthology.orchestra.run.smithi133.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:01:56.359 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-09-10T18:01:56.359 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:01:56.359 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:01:56.359 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:01:56.359 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:01:56.360 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-09-10T18:01:56.360 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-09-10T18:01:56.360 INFO:teuthology.orchestra.run.smithi133.stdout:Install 4 Packages 2024-09-10T18:01:56.360 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:01:56.360 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 79 M 2024-09-10T18:01:56.360 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 126 M 2024-09-10T18:01:56.360 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-09-10T18:01:56.561 INFO:teuthology.orchestra.run.smithi133.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 273 kB/s | 23 kB 00:00 2024-09-10T18:01:56.986 INFO:teuthology.orchestra.run.smithi133.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-10T18:01:57.270 INFO:teuthology.orchestra.run.smithi133.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 40 MB/s | 29 MB 00:00 2024-09-10T18:01:57.314 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-10T18:01:57.314 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-10T18:01:57.779 INFO:teuthology.orchestra.run.smithi133.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 27 MB/s | 35 MB 00:01 2024-09-10T18:01:57.779 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:01:57.779 INFO:teuthology.orchestra.run.smithi133.stdout:Total 55 MB/s | 79 MB 00:01 2024-09-10T18:01:58.457 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-09-10T18:01:58.809 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-09-10T18:01:58.810 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-09-10T18:01:59.761 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-09-10T18:01:59.761 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-09-10T18:01:59.784 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-10T18:02:00.410 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:02:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:02:02.217 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-09-10T18:02:02.880 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:02:02.962 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:02:03.273 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:02:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:02:05.766 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:02:05.896 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:02:08.279 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:02:08.380 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:02:10.716 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:02:10.764 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:02:13.218 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:02:15.596 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:02:51.430 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:02:51.469 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:02:51.742 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:02:51.742 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:02:51.742 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:02:51.742 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:02:52.569 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:02:52.569 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:02:52.569 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-10T18:02:52.569 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:02:52.569 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:02:52.569 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:02:52.569 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:02:52.569 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:02:52.569 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-10T18:02:52.928 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:02:53.021 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:02:53.022 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:02:53.022 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:02:53.022 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:02:53.022 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:02:53.023 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T18:02:53.023 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:02:53.023 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:02:53.081 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:02:53.082 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:02:53.082 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -qi grub2-tools 2024-09-10T18:02:53.117 INFO:teuthology.orchestra.run.smithi099.stdout:Name : grub2-tools 2024-09-10T18:02:53.117 INFO:teuthology.orchestra.run.smithi099.stdout:Epoch : 1 2024-09-10T18:02:53.117 INFO:teuthology.orchestra.run.smithi099.stdout:Version : 2.06 2024-09-10T18:02:53.117 INFO:teuthology.orchestra.run.smithi099.stdout:Release : 82.el9 2024-09-10T18:02:53.117 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: x86_64 2024-09-10T18:02:53.117 INFO:teuthology.orchestra.run.smithi099.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:02:53.118 INFO:teuthology.orchestra.run.smithi099.stdout:Group : Unspecified 2024-09-10T18:02:53.118 INFO:teuthology.orchestra.run.smithi099.stdout:Size : 8274670 2024-09-10T18:02:53.118 INFO:teuthology.orchestra.run.smithi099.stdout:License : GPLv3+ 2024-09-10T18:02:53.118 INFO:teuthology.orchestra.run.smithi099.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:02:53.118 INFO:teuthology.orchestra.run.smithi099.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:02:53.118 INFO:teuthology.orchestra.run.smithi099.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:02:53.118 INFO:teuthology.orchestra.run.smithi099.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:02:53.118 INFO:teuthology.orchestra.run.smithi099.stdout:Packager : builder@centos.org 2024-09-10T18:02:53.118 INFO:teuthology.orchestra.run.smithi099.stdout:Vendor : CentOS 2024-09-10T18:02:53.118 INFO:teuthology.orchestra.run.smithi099.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:02:53.118 INFO:teuthology.orchestra.run.smithi099.stdout:Summary : Support tools for GRUB. 2024-09-10T18:02:53.118 INFO:teuthology.orchestra.run.smithi099.stdout:Description : 2024-09-10T18:02:53.118 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:02:53.119 INFO:teuthology.orchestra.run.smithi099.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:02:53.119 INFO:teuthology.orchestra.run.smithi099.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:02:53.119 INFO:teuthology.orchestra.run.smithi099.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:02:53.119 INFO:teuthology.orchestra.run.smithi099.stdout:hardware devices. 2024-09-10T18:02:53.119 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:02:53.119 INFO:teuthology.orchestra.run.smithi099.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:02:53.120 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:02:53.120 INFO:teuthology.task.kernel:Updating grub on smithi099 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:02:53.120 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:02:53.718 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:02:53.756 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2024-09-10T18:02:53.758 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:02:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:02:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:02:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:02:54.092 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:02:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:02:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:02:54.938 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-09-10T18:02:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:02:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:02:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:02:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:02:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:02:54.938 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-09-10T18:02:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:02:55.088 INFO:teuthology.orchestra.run.smithi099.stderr:done 2024-09-10T18:02:55.090 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-09-10T18:02:55.106 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.D9FHFCbuYO 2024-09-10T18:02:55.106 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.D9FHFCbuYO 2024-09-10T18:02:55.173 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.D9FHFCbuYO 2024-09-10T18:02:55.268 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:02:55.314 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.D9FHFCbuYO is 6KB 2024-09-10T18:02:55.361 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:02:55.361 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:02:55.361 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:02:55.361 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:02:55.361 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:02:55.361 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-10T18:02:55.361 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:02:55.361 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:02:55.365 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.D9FHFCbuYO 2024-09-10T18:02:55.380 DEBUG:teuthology.orchestra.run.smithi099:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:02:55.420 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:02:55.421 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:02:55.421 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -qi grub2-tools 2024-09-10T18:02:55.444 INFO:teuthology.orchestra.run.smithi099.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:02:55.444 INFO:teuthology.orchestra.run.smithi099.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:02:55.444 INFO:teuthology.orchestra.run.smithi099.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:02:55.444 INFO:teuthology.orchestra.run.smithi099.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:02:55.445 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:02:55.457 INFO:teuthology.orchestra.run.smithi133.stdout:Name : grub2-tools 2024-09-10T18:02:55.457 INFO:teuthology.orchestra.run.smithi133.stdout:Epoch : 1 2024-09-10T18:02:55.457 INFO:teuthology.orchestra.run.smithi133.stdout:Version : 2.06 2024-09-10T18:02:55.457 INFO:teuthology.orchestra.run.smithi133.stdout:Release : 82.el9 2024-09-10T18:02:55.457 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: x86_64 2024-09-10T18:02:55.457 INFO:teuthology.orchestra.run.smithi133.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:02:55.457 INFO:teuthology.orchestra.run.smithi133.stdout:Group : Unspecified 2024-09-10T18:02:55.458 INFO:teuthology.orchestra.run.smithi133.stdout:Size : 8274670 2024-09-10T18:02:55.458 INFO:teuthology.orchestra.run.smithi133.stdout:License : GPLv3+ 2024-09-10T18:02:55.458 INFO:teuthology.orchestra.run.smithi133.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:02:55.458 INFO:teuthology.orchestra.run.smithi133.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:02:55.458 INFO:teuthology.orchestra.run.smithi133.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:02:55.458 INFO:teuthology.orchestra.run.smithi133.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:02:55.459 INFO:teuthology.orchestra.run.smithi133.stdout:Packager : builder@centos.org 2024-09-10T18:02:55.459 INFO:teuthology.orchestra.run.smithi133.stdout:Vendor : CentOS 2024-09-10T18:02:55.459 INFO:teuthology.orchestra.run.smithi133.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:02:55.459 INFO:teuthology.orchestra.run.smithi133.stdout:Summary : Support tools for GRUB. 2024-09-10T18:02:55.459 INFO:teuthology.orchestra.run.smithi133.stdout:Description : 2024-09-10T18:02:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:02:55.460 INFO:teuthology.orchestra.run.smithi133.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:02:55.460 INFO:teuthology.orchestra.run.smithi133.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:02:55.460 INFO:teuthology.orchestra.run.smithi133.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:02:55.460 INFO:teuthology.orchestra.run.smithi133.stdout:hardware devices. 2024-09-10T18:02:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:02:55.460 INFO:teuthology.orchestra.run.smithi133.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:02:55.462 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:02:55.462 INFO:teuthology.task.kernel:Updating grub on smithi133 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:02:55.462 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:02:55.604 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2024-09-10T18:02:56.100 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2024-09-10T18:02:57.535 INFO:teuthology.orchestra.run.smithi133.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:02:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:done 2024-09-10T18:02:57.566 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-09-10T18:02:57.582 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.AlmvY7hfml 2024-09-10T18:02:57.582 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AlmvY7hfml 2024-09-10T18:02:57.649 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.AlmvY7hfml 2024-09-10T18:02:57.788 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.AlmvY7hfml is 6KB 2024-09-10T18:02:57.836 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.AlmvY7hfml 2024-09-10T18:02:57.851 DEBUG:teuthology.orchestra.run.smithi133:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:02:57.914 INFO:teuthology.orchestra.run.smithi133.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:02:57.914 INFO:teuthology.orchestra.run.smithi133.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:02:57.914 INFO:teuthology.orchestra.run.smithi133.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:02:57.915 INFO:teuthology.orchestra.run.smithi133.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:02:57.917 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:02:58.056 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2024-09-10T18:03:25.629 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T18:03:25.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T18:03:25.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:03:28.061 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T18:03:28.061 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T18:03:28.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:03:44.061 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-10T18:03:46.621 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-09-10T18:03:53.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T18:03:53.070 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:03:55.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T18:03:55.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:03:56.125 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-10T18:03:58.685 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-09-10T18:04:08.127 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T18:04:08.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:04:10.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T18:04:10.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:04:26.561 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-10T18:04:41.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T18:04:41.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:04:41.976 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-10T18:04:42.327 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T18:04:42.327 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:04:42.328 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-09-10T18:04:42.385 INFO:teuthology.orchestra.run.smithi099.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:04:42.385 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:04:42.385 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:04:42.385 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:04:42.418 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-09-10T18:04:42.910 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T18:04:42.910 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:04:42.910 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-09-10T18:04:42.925 INFO:teuthology.orchestra.run.smithi133.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:04:42.925 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:04:42.925 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:04:42.925 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:04:43.386 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T18:04:43.386 DEBUG:teuthology.orchestra.run.smithi099:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:04:43.537 INFO:teuthology.orchestra.run.smithi099.stdout:ttyS1 2024-09-10T18:04:43.576 DEBUG:teuthology.parallel:result is None 2024-09-10T18:04:43.926 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T18:04:43.927 DEBUG:teuthology.orchestra.run.smithi133:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:04:44.035 INFO:teuthology.orchestra.run.smithi133.stdout:ttyS1 2024-09-10T18:04:44.054 DEBUG:teuthology.parallel:result is None 2024-09-10T18:04:44.054 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T18:04:44.062 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T18:04:44.062 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:04:44.064 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:04:44.079 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T18:04:44.085 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T18:04:44.091 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T18:04:44.092 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:04:44.124 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:04:44.159 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T18:04:44.165 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T18:04:44.165 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:04:44.194 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:04:44.225 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:04:44.225 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:04:44.235 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:04:44.237 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:04:44.238 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T18:04:44.245 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T18:04:44.245 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:04:44.284 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:04:44.490 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T18:04:44.499 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T18:04:44.500 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:04:44.502 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:04:44.519 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:04:44.599 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:04:44.655 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:04:44.729 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:04:44.777 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:04:44.777 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:04:44.843 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:04:44.880 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:04:44.928 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:04:45.007 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:04:45.047 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T18:04:45.047 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:04:45.112 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-09-10T18:04:45.114 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-09-10T18:04:45.159 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:04:45.207 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:04:45.656 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T18:04:45.664 INFO:teuthology.task.internal:Starting timer... 2024-09-10T18:04:45.664 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T18:04:45.674 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T18:04:45.683 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T18:04:45.683 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2024-09-10T18:04:45.747 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2024-09-10T18:04:45.749 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2024-09-10T18:04:45.796 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2024-09-10T18:04:45.798 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T18:04:45.798 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2024-09-10T18:04:45.843 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2024-09-10T18:04:45.844 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2024-09-10T18:04:45.878 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2024-09-10T18:04:45.878 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi133.front.sepia.ceph.com': 'permissive'} 2024-09-10T18:04:45.878 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:04:45.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:04:45.903 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:04:45.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:04:45.943 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T18:04:45.943 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:04:45.971 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:04:46.023 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T18:04:46.032 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T18:04:46.035 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T18:04:46.145 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T18:04:46.155 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T18:04:46.156 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi099.front.sepia.ceph.com,smithi133.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T18:08:29.268 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi133.front.sepia.ceph.com')] 2024-09-10T18:08:29.269 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T18:08:29.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:08:29.348 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-10T18:08:29.423 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T18:08:29.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T18:08:29.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:08:29.502 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-09-10T18:08:29.586 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T18:08:29.587 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T18:08:29.598 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T18:08:29.599 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:08:29.599 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:08:29.602 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:08:29.602 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:08:29.628 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:08:29.642 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:08:29.660 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:08:29.666 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2024-09-10T18:08:29.676 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:08:29.677 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2024-09-10T18:08:29.691 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:08:29.703 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2024-09-10T18:08:29.705 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:08:29.714 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2024-09-10T18:08:29.730 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:08:29.744 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:08:29.758 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:08:29.761 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:08:29.761 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-09-10T18:08:29.762 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:08:29.762 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:08:29.762 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:08:29.762 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:08:29.799 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:08:29.810 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:08:29.810 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-09-10T18:08:29.810 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:08:29.811 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:08:29.811 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:08:29.811 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:08:29.811 INFO:teuthology.run_tasks:Running task pexec... 2024-09-10T18:08:29.823 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-10T18:08:29.823 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T18:08:29.823 DEBUG:teuthology.orchestra.run.smithi099:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:08:29.824 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T18:08:29.824 DEBUG:teuthology.orchestra.run.smithi133:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:08:30.172 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-10T18:08:30.189 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-10T18:08:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2024-09-10T18:08:30.190 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-10T18:08:30.190 INFO:teuthology.orchestra.run.smithi099.stdout:Removing: 2024-09-10T18:08:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:08:30.190 INFO:teuthology.orchestra.run.smithi099.stdout:Removing unused dependencies: 2024-09-10T18:08:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:08:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:08:30.190 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-10T18:08:30.190 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-10T18:08:30.190 INFO:teuthology.orchestra.run.smithi099.stdout:Remove 2 Packages 2024-09-10T18:08:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:08:30.190 INFO:teuthology.orchestra.run.smithi099.stdout:Freed space: 5.5 M 2024-09-10T18:08:30.190 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-10T18:08:30.194 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-10T18:08:30.194 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-10T18:08:30.202 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-09-10T18:08:30.204 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-09-10T18:08:30.204 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2024-09-10T18:08:30.204 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-09-10T18:08:30.204 INFO:teuthology.orchestra.run.smithi133.stdout:Removing: 2024-09-10T18:08:30.204 INFO:teuthology.orchestra.run.smithi133.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:08:30.204 INFO:teuthology.orchestra.run.smithi133.stdout:Removing unused dependencies: 2024-09-10T18:08:30.204 INFO:teuthology.orchestra.run.smithi133.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:08:30.204 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:08:30.204 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-09-10T18:08:30.204 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-09-10T18:08:30.204 INFO:teuthology.orchestra.run.smithi133.stdout:Remove 2 Packages 2024-09-10T18:08:30.205 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:08:30.205 INFO:teuthology.orchestra.run.smithi133.stdout:Freed space: 5.5 M 2024-09-10T18:08:30.205 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-09-10T18:08:30.208 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-09-10T18:08:30.209 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-09-10T18:08:30.256 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-10T18:08:30.256 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-10T18:08:30.287 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-09-10T18:08:30.287 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-09-10T18:08:30.404 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-10T18:08:30.431 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-09-10T18:08:30.484 INFO:teuthology.orchestra.run.smithi099.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:08:30.507 INFO:teuthology.orchestra.run.smithi133.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:08:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:08:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:08:31.055 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:08:31.055 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:08:31.069 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:08:31.069 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:08:31.280 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:08:31.280 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:08:31.280 INFO:teuthology.orchestra.run.smithi099.stdout:Removed: 2024-09-10T18:08:31.280 INFO:teuthology.orchestra.run.smithi099.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:08:31.281 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:08:31.281 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-10T18:08:31.300 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:08:31.300 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:08:31.300 INFO:teuthology.orchestra.run.smithi133.stdout:Removed: 2024-09-10T18:08:31.300 INFO:teuthology.orchestra.run.smithi133.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:08:31.300 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:08:31.300 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-09-10T18:08:32.107 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:48 ago on Tue 10 Sep 2024 06:07:43 PM UTC. 2024-09-10T18:08:32.169 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:00:49 ago on Tue 10 Sep 2024 06:07:42 PM UTC. 2024-09-10T18:08:32.249 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-10T18:08:32.250 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-10T18:08:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2024-09-10T18:08:32.250 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-10T18:08:32.250 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-10T18:08:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:08:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:08:32.251 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-10T18:08:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:08:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:08:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:08:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:08:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:08:32.251 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-10T18:08:32.251 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-10T18:08:32.251 INFO:teuthology.orchestra.run.smithi099.stdout:Install 6 Packages 2024-09-10T18:08:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:08:32.257 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 2.0 M 2024-09-10T18:08:32.257 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 9.2 M 2024-09-10T18:08:32.257 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-10T18:08:32.306 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-09-10T18:08:32.306 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-09-10T18:08:32.306 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2024-09-10T18:08:32.306 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-09-10T18:08:32.306 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-09-10T18:08:32.307 INFO:teuthology.orchestra.run.smithi133.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:08:32.307 INFO:teuthology.orchestra.run.smithi133.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:08:32.307 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-09-10T18:08:32.307 INFO:teuthology.orchestra.run.smithi133.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:08:32.307 INFO:teuthology.orchestra.run.smithi133.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:08:32.307 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:08:32.307 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:08:32.307 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:08:32.307 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-09-10T18:08:32.307 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-09-10T18:08:32.307 INFO:teuthology.orchestra.run.smithi133.stdout:Install 6 Packages 2024-09-10T18:08:32.307 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:08:32.313 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 2.0 M 2024-09-10T18:08:32.313 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 9.2 M 2024-09-10T18:08:32.313 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-09-10T18:08:32.580 INFO:teuthology.orchestra.run.smithi133.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 303 kB/s | 44 kB 00:00 2024-09-10T18:08:32.605 INFO:teuthology.orchestra.run.smithi133.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 617 kB/s | 106 kB 00:00 2024-09-10T18:08:32.651 INFO:teuthology.orchestra.run.smithi099.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 388 kB/s | 44 kB 00:00 2024-09-10T18:08:32.664 INFO:teuthology.orchestra.run.smithi133.stdout:(3/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.4 MB/s | 84 kB 00:00 2024-09-10T18:08:32.676 INFO:teuthology.orchestra.run.smithi099.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 758 kB/s | 106 kB 00:00 2024-09-10T18:08:32.689 INFO:teuthology.orchestra.run.smithi133.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 660 kB/s | 72 kB 00:00 2024-09-10T18:08:32.723 INFO:teuthology.orchestra.run.smithi133.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.1 MB/s | 906 kB 00:00 2024-09-10T18:08:32.726 INFO:teuthology.orchestra.run.smithi099.stdout:(3/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.7 MB/s | 84 kB 00:00 2024-09-10T18:08:32.752 INFO:teuthology.orchestra.run.smithi099.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 711 kB/s | 72 kB 00:00 2024-09-10T18:08:32.815 INFO:teuthology.orchestra.run.smithi133.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 5.4 MB/s | 837 kB 00:00 2024-09-10T18:08:32.815 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:08:32.815 INFO:teuthology.orchestra.run.smithi133.stdout:Total 4.0 MB/s | 2.0 MB 00:00 2024-09-10T18:08:32.852 INFO:teuthology.orchestra.run.smithi099.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.8 MB/s | 906 kB 00:00 2024-09-10T18:08:32.889 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-09-10T18:08:32.903 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-09-10T18:08:32.903 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-09-10T18:08:32.910 INFO:teuthology.orchestra.run.smithi099.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.4 MB/s | 837 kB 00:00 2024-09-10T18:08:32.910 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:08:32.911 INFO:teuthology.orchestra.run.smithi099.stdout:Total 3.1 MB/s | 2.0 MB 00:00 2024-09-10T18:08:32.989 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-10T18:08:33.003 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-10T18:08:33.003 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-10T18:08:33.190 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-09-10T18:08:33.190 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-09-10T18:08:33.293 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-10T18:08:33.293 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-10T18:08:33.620 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-09-10T18:08:33.707 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-10T18:08:33.708 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-10T18:08:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-10T18:08:33.780 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-10T18:08:33.796 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-10T18:08:33.830 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-10T18:08:33.859 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-10T18:08:33.986 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-10T18:08:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:08:34.075 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-10T18:08:34.107 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:08:34.455 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:08:34.481 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:08:34.546 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:08:34.585 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:08:35.423 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:08:35.423 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-10T18:08:35.423 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-10T18:08:35.423 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-10T18:08:35.423 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-10T18:08:35.423 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-10T18:08:35.593 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:08:35.593 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-10T18:08:35.593 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-10T18:08:35.594 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-10T18:08:35.594 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-10T18:08:35.594 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-10T18:08:35.752 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-10T18:08:35.753 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:08:35.753 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-09-10T18:08:35.753 INFO:teuthology.orchestra.run.smithi133.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:08:35.753 INFO:teuthology.orchestra.run.smithi133.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:08:35.753 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:08:35.753 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:08:35.753 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-09-10T18:08:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-10T18:08:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:08:35.937 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-10T18:08:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:08:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:08:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:08:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:08:35.938 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-10T18:08:36.063 DEBUG:teuthology.parallel:result is None 2024-09-10T18:08:36.266 DEBUG:teuthology.parallel:result is None 2024-09-10T18:08:36.267 INFO:teuthology.run_tasks:Running task install... 2024-09-10T18:08:36.275 DEBUG:teuthology.task.install:project ceph 2024-09-10T18:08:36.275 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}, 'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'}} 2024-09-10T18:08:36.275 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}, 'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'} 2024-09-10T18:08:36.275 INFO:teuthology.task.install:Using flavor: default 2024-09-10T18:08:36.281 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T18:08:36.281 INFO:teuthology.task.install:extra packages: [] 2024-09-10T18:08:36.282 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T18:08:36.282 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:08:36.285 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T18:08:36.285 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:08:36.426 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T18:08:36.427 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T18:08:36.445 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T18:08:36.448 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T18:08:36.645 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T18:08:36.645 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T18:08:36.645 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T18:08:36.648 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T18:08:36.648 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:08:36.648 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T18:08:36.677 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest, python3-pytest, python3-pytest on remote rpm x86_64 2024-09-10T18:08:36.677 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T18:08:36.678 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest, python3-pytest, python3-pytest, python3-pytest on remote rpm x86_64 2024-09-10T18:08:36.678 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T18:08:36.743 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T18:08:36.748 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T18:08:36.818 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T18:08:36.826 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T18:08:36.888 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2024-09-10T18:08:36.890 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2024-09-10T18:08:36.892 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2024-09-10T18:08:36.894 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2024-09-10T18:08:37.116 INFO:teuthology.orchestra.run.smithi133.stdout:45 files removed 2024-09-10T18:08:37.118 INFO:teuthology.orchestra.run.smithi099.stdout:45 files removed 2024-09-10T18:08:37.144 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest python3-pytest python3-pytest python3-pytest 2024-09-10T18:08:37.145 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest python3-pytest python3-pytest python3-pytest 2024-09-10T18:08:38.120 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 111 kB/s | 77 kB 00:00 2024-09-10T18:08:38.123 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 111 kB/s | 77 kB 00:00 2024-09-10T18:08:38.737 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 19 kB/s | 11 kB 00:00 2024-09-10T18:08:38.749 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 18 kB/s | 11 kB 00:00 2024-09-10T18:08:39.041 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 7.0 kB/s | 1.8 kB 00:00 2024-09-10T18:08:39.046 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 6.7 kB/s | 1.8 kB 00:00 2024-09-10T18:08:39.799 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-10T18:08:41.338 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.2 MB 00:02 2024-09-10T18:08:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-10T18:08:48.825 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.5 MB 00:03 2024-09-10T18:08:50.215 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-10T18:08:50.267 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 19 kB 00:00 2024-09-10T18:08:51.205 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-10T18:08:54.991 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.5 MB 00:00 2024-09-10T18:08:56.652 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T18:08:56.699 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-10T18:08:57.445 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-10T18:08:58.988 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T18:08:58.988 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T18:08:59.063 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-10T18:08:59.069 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-09-10T18:08:59.069 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-10T18:08:59.069 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-09-10T18:08:59.069 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-10T18:08:59.070 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T18:08:59.070 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T18:08:59.070 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T18:08:59.070 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T18:08:59.070 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T18:08:59.070 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T18:08:59.070 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T18:08:59.070 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T18:08:59.070 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T18:08:59.070 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T18:08:59.070 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T18:08:59.070 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T18:08:59.071 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T18:08:59.071 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:08:59.071 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T18:08:59.071 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T18:08:59.071 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T18:08:59.071 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytest noarch 6.2.2-7.el9 appstream 519 k 2024-09-10T18:08:59.071 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T18:08:59.071 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T18:08:59.071 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T18:08:59.071 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T18:08:59.071 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:08:59.071 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T18:08:59.072 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2024-09-10T18:08:59.072 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T18:08:59.072 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:08:59.072 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-10T18:08:59.072 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T18:08:59.072 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T18:08:59.072 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T18:08:59.072 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T18:08:59.072 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T18:08:59.072 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T18:08:59.072 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T18:08:59.073 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:08:59.073 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T18:08:59.073 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T18:08:59.073 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T18:08:59.073 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T18:08:59.073 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T18:08:59.073 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T18:08:59.073 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T18:08:59.073 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T18:08:59.073 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T18:08:59.073 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T18:08:59.073 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T18:08:59.074 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T18:08:59.074 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T18:08:59.074 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T18:08:59.074 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T18:08:59.074 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T18:08:59.074 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T18:08:59.074 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T18:08:59.074 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T18:08:59.074 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T18:08:59.074 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T18:08:59.074 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T18:08:59.075 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T18:08:59.075 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T18:08:59.075 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T18:08:59.075 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T18:08:59.075 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T18:08:59.075 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T18:08:59.075 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T18:08:59.075 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T18:08:59.075 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T18:08:59.075 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T18:08:59.075 INFO:teuthology.orchestra.run.smithi099.stdout: python3-iniconfig noarch 1.1.1-7.el9 appstream 17 k 2024-09-10T18:08:59.075 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T18:08:59.076 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T18:08:59.076 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T18:08:59.076 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T18:08:59.076 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T18:08:59.076 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T18:08:59.076 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T18:08:59.076 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T18:08:59.076 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T18:08:59.076 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T18:08:59.076 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T18:08:59.076 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T18:08:59.076 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T18:08:59.077 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T18:08:59.077 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T18:08:59.077 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T18:08:59.077 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T18:08:59.077 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T18:08:59.077 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T18:08:59.077 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T18:08:59.077 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pluggy noarch 0.13.1-7.el9 appstream 41 k 2024-09-10T18:08:59.077 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T18:08:59.077 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T18:08:59.077 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T18:08:59.077 INFO:teuthology.orchestra.run.smithi099.stdout: python3-py noarch 1.10.0-6.el9 appstream 477 k 2024-09-10T18:08:59.078 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T18:08:59.078 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T18:08:59.078 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T18:08:59.078 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T18:08:59.078 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T18:08:59.078 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T18:08:59.078 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T18:08:59.078 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T18:08:59.078 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T18:08:59.078 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T18:08:59.078 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T18:08:59.078 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T18:08:59.078 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T18:08:59.079 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T18:08:59.079 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T18:08:59.079 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T18:08:59.079 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T18:08:59.079 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T18:08:59.079 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T18:08:59.079 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T18:08:59.079 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T18:08:59.079 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T18:08:59.079 INFO:teuthology.orchestra.run.smithi099.stdout:Installing weak dependencies: 2024-09-10T18:08:59.079 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T18:08:59.079 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:08:59.080 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-10T18:08:59.080 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-09-10T18:08:59.080 INFO:teuthology.orchestra.run.smithi099.stdout:Install 113 Packages 2024-09-10T18:08:59.080 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 2 Packages 2024-09-10T18:08:59.080 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:08:59.080 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 191 M 2024-09-10T18:08:59.080 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-10T18:08:59.528 INFO:teuthology.orchestra.run.smithi099.stdout:(1/115): ceph-17.2.7-1606.g5be57395.el9.x86_64. 43 kB/s | 6.5 kB 00:00 2024-09-10T18:08:59.828 INFO:teuthology.orchestra.run.smithi099.stdout:(2/115): ceph-fuse-17.2.7-1606.g5be57395.el9.x8 2.6 MB/s | 813 kB 00:00 2024-09-10T18:09:00.545 INFO:teuthology.orchestra.run.smithi099.stdout:(3/115): ceph-common-17.2.7-1606.g5be57395.el9. 19 MB/s | 22 MB 00:01 2024-09-10T18:09:00.570 INFO:teuthology.orchestra.run.smithi099.stdout:(4/115): ceph-immutable-object-cache-17.2.7-160 185 kB/s | 137 kB 00:00 2024-09-10T18:09:00.653 INFO:teuthology.orchestra.run.smithi099.stdout:(5/115): ceph-base-17.2.7-1606.g5be57395.el9.x8 3.9 MB/s | 5.0 MB 00:01 2024-09-10T18:09:00.720 INFO:teuthology.orchestra.run.smithi099.stdout:(6/115): ceph-mds-17.2.7-1606.g5be57395.el9.x86 12 MB/s | 2.1 MB 00:00 2024-09-10T18:09:00.929 INFO:teuthology.orchestra.run.smithi099.stdout:(7/115): ceph-mon-17.2.7-1606.g5be57395.el9.x86 14 MB/s | 3.9 MB 00:00 2024-09-10T18:09:01.421 INFO:teuthology.orchestra.run.smithi099.stdout:(8/115): ceph-osd-17.2.7-1606.g5be57395.el9.x86 22 MB/s | 15 MB 00:00 2024-09-10T18:09:01.463 INFO:teuthology.orchestra.run.smithi099.stdout:(9/115): ceph-mgr-17.2.7-1606.g5be57395.el9.x86 1.6 MB/s | 1.4 MB 00:00 2024-09-10T18:09:01.880 INFO:teuthology.orchestra.run.smithi099.stdout:(10/115): ceph-radosgw-17.2.7-1606.g5be57395.el 15 MB/s | 14 MB 00:00 2024-09-10T18:09:01.905 INFO:teuthology.orchestra.run.smithi099.stdout:(11/115): ceph-selinux-17.2.7-1606.g5be57395.el 52 kB/s | 25 kB 00:00 2024-09-10T18:09:02.297 INFO:teuthology.orchestra.run.smithi099.stdout:(12/115): libcephfs-devel-17.2.7-1606.g5be57395 60 kB/s | 25 kB 00:00 2024-09-10T18:09:02.372 INFO:teuthology.orchestra.run.smithi099.stdout:(13/115): libcephfs2-17.2.7-1606.g5be57395.el9. 1.3 MB/s | 637 kB 00:00 2024-09-10T18:09:02.397 INFO:teuthology.orchestra.run.smithi099.stdout:(14/115): libcephsqlite-17.2.7-1606.g5be57395.e 1.6 MB/s | 163 kB 00:00 2024-09-10T18:09:02.439 INFO:teuthology.orchestra.run.smithi099.stdout:(15/115): librados-devel-17.2.7-1606.g5be57395. 1.9 MB/s | 126 kB 00:00 2024-09-10T18:09:02.489 INFO:teuthology.orchestra.run.smithi099.stdout:(16/115): libradosstriper1-17.2.7-1606.g5be5739 4.9 MB/s | 458 kB 00:00 2024-09-10T18:09:02.765 INFO:teuthology.orchestra.run.smithi099.stdout:(17/115): librgw2-17.2.7-1606.g5be57395.el9.x86 15 MB/s | 4.9 MB 00:00 2024-09-10T18:09:02.790 INFO:teuthology.orchestra.run.smithi099.stdout:(18/115): python3-ceph-argparse-17.2.7-1606.g5b 150 kB/s | 45 kB 00:00 2024-09-10T18:09:02.823 INFO:teuthology.orchestra.run.smithi099.stdout:(19/115): python3-ceph-common-17.2.7-1606.g5be5 2.0 MB/s | 120 kB 00:00 2024-09-10T18:09:02.857 INFO:teuthology.orchestra.run.smithi099.stdout:(20/115): python3-cephfs-17.2.7-1606.g5be57395. 2.3 MB/s | 156 kB 00:00 2024-09-10T18:09:02.890 INFO:teuthology.orchestra.run.smithi099.stdout:(21/115): python3-rados-17.2.7-1606.g5be57395.e 4.7 MB/s | 320 kB 00:00 2024-09-10T18:09:02.942 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-10T18:09:02.949 INFO:teuthology.orchestra.run.smithi099.stdout:(22/115): python3-rbd-17.2.7-1606.g5be57395.el9 3.1 MB/s | 295 kB 00:00 2024-09-10T18:09:04.291 INFO:teuthology.orchestra.run.smithi099.stdout:(23/115): ceph-test-17.2.7-1606.g5be57395.el9.x 17 MB/s | 48 MB 00:02 2024-09-10T18:09:04.316 INFO:teuthology.orchestra.run.smithi099.stdout:(24/115): rbd-fuse-17.2.7-1606.g5be57395.el9.x8 60 kB/s | 82 kB 00:01 2024-09-10T18:09:04.341 INFO:teuthology.orchestra.run.smithi099.stdout:(25/115): python3-rgw-17.2.7-1606.g5be57395.el9 68 kB/s | 99 kB 00:01 2024-09-10T18:09:04.417 INFO:teuthology.orchestra.run.smithi099.stdout:(26/115): rbd-nbd-17.2.7-1606.g5be57395.el9.x86 1.6 MB/s | 165 kB 00:00 2024-09-10T18:09:04.475 INFO:teuthology.orchestra.run.smithi099.stdout:(27/115): rbd-mirror-17.2.7-1606.g5be57395.el9. 16 MB/s | 2.9 MB 00:00 2024-09-10T18:09:04.501 INFO:teuthology.orchestra.run.smithi099.stdout:(28/115): ceph-grafana-dashboards-17.2.7-1606.g 154 kB/s | 24 kB 00:00 2024-09-10T18:09:04.517 INFO:teuthology.orchestra.run.smithi099.stdout:(29/115): ceph-mgr-cephadm-17.2.7-1606.g5be5739 1.2 MB/s | 124 kB 00:00 2024-09-10T18:09:04.626 INFO:teuthology.orchestra.run.smithi099.stdout:(30/115): ceph-mgr-dashboard-17.2.7-1606.g5be57 22 MB/s | 3.4 MB 00:00 2024-09-10T18:09:04.834 INFO:teuthology.orchestra.run.smithi099.stdout:(31/115): ceph-mgr-modules-core-17.2.7-1606.g5b 758 kB/s | 240 kB 00:00 2024-09-10T18:09:04.860 INFO:teuthology.orchestra.run.smithi099.stdout:(32/115): ceph-mgr-rook-17.2.7-1606.g5be57395.e 203 kB/s | 47 kB 00:00 2024-09-10T18:09:04.977 INFO:teuthology.orchestra.run.smithi099.stdout:(33/115): ceph-mgr-diskprediction-local-17.2.7- 16 MB/s | 7.4 MB 00:00 2024-09-10T18:09:05.280 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T18:09:05.281 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T18:09:05.349 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-09-10T18:09:05.356 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-09-10T18:09:05.356 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-09-10T18:09:05.356 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-09-10T18:09:05.356 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-09-10T18:09:05.356 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T18:09:05.356 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T18:09:05.357 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T18:09:05.357 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T18:09:05.357 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T18:09:05.357 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T18:09:05.357 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T18:09:05.357 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T18:09:05.357 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T18:09:05.357 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T18:09:05.357 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T18:09:05.358 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T18:09:05.358 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T18:09:05.358 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:09:05.358 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T18:09:05.358 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T18:09:05.358 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T18:09:05.358 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytest noarch 6.2.2-7.el9 appstream 519 k 2024-09-10T18:09:05.358 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T18:09:05.358 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T18:09:05.358 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T18:09:05.358 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T18:09:05.358 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:09:05.359 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T18:09:05.359 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2024-09-10T18:09:05.359 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T18:09:05.359 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:09:05.359 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-09-10T18:09:05.359 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T18:09:05.359 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T18:09:05.359 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T18:09:05.359 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T18:09:05.359 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T18:09:05.359 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T18:09:05.359 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T18:09:05.359 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:09:05.360 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T18:09:05.360 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T18:09:05.360 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T18:09:05.360 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T18:09:05.360 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T18:09:05.360 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T18:09:05.360 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T18:09:05.360 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T18:09:05.360 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T18:09:05.360 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T18:09:05.360 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T18:09:05.361 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T18:09:05.361 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T18:09:05.361 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T18:09:05.361 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T18:09:05.361 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T18:09:05.361 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T18:09:05.361 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T18:09:05.361 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T18:09:05.361 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T18:09:05.362 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T18:09:05.362 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T18:09:05.362 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T18:09:05.362 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T18:09:05.362 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T18:09:05.362 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T18:09:05.362 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T18:09:05.362 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T18:09:05.362 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T18:09:05.362 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T18:09:05.362 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T18:09:05.362 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T18:09:05.363 INFO:teuthology.orchestra.run.smithi133.stdout: python3-iniconfig noarch 1.1.1-7.el9 appstream 17 k 2024-09-10T18:09:05.363 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T18:09:05.363 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T18:09:05.363 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T18:09:05.363 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T18:09:05.363 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T18:09:05.363 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T18:09:05.363 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T18:09:05.363 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T18:09:05.363 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T18:09:05.363 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T18:09:05.363 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T18:09:05.364 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T18:09:05.364 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T18:09:05.364 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T18:09:05.364 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T18:09:05.364 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T18:09:05.364 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T18:09:05.364 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T18:09:05.364 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T18:09:05.364 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T18:09:05.364 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pluggy noarch 0.13.1-7.el9 appstream 41 k 2024-09-10T18:09:05.364 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T18:09:05.364 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T18:09:05.364 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T18:09:05.365 INFO:teuthology.orchestra.run.smithi133.stdout: python3-py noarch 1.10.0-6.el9 appstream 477 k 2024-09-10T18:09:05.365 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T18:09:05.365 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T18:09:05.365 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T18:09:05.365 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T18:09:05.365 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T18:09:05.365 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T18:09:05.365 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T18:09:05.365 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T18:09:05.366 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T18:09:05.366 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T18:09:05.366 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T18:09:05.366 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T18:09:05.366 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T18:09:05.366 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T18:09:05.366 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T18:09:05.366 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T18:09:05.366 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T18:09:05.366 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T18:09:05.366 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T18:09:05.366 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T18:09:05.366 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T18:09:05.367 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T18:09:05.367 INFO:teuthology.orchestra.run.smithi133.stdout:Installing weak dependencies: 2024-09-10T18:09:05.367 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T18:09:05.367 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:09:05.367 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-09-10T18:09:05.367 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-09-10T18:09:05.367 INFO:teuthology.orchestra.run.smithi133.stdout:Install 113 Packages 2024-09-10T18:09:05.367 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 2 Packages 2024-09-10T18:09:05.367 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:09:05.368 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 191 M 2024-09-10T18:09:05.368 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-09-10T18:09:05.760 INFO:teuthology.orchestra.run.smithi099.stdout:(34/115): ceph-prometheus-alerts-17.2.7-1606.g5 16 kB/s | 15 kB 00:00 2024-09-10T18:09:05.819 INFO:teuthology.orchestra.run.smithi099.stdout:(35/115): ceph-volume-17.2.7-1606.g5be57395.el9 281 kB/s | 270 kB 00:00 2024-09-10T18:09:05.827 INFO:teuthology.orchestra.run.smithi133.stdout:(1/115): ceph-17.2.7-1606.g5be57395.el9.x86_64. 47 kB/s | 6.5 kB 00:00 2024-09-10T18:09:05.844 INFO:teuthology.orchestra.run.smithi099.stdout:(36/115): cephadm-17.2.7-1606.g5be57395.el9.noa 95 kB/s | 83 kB 00:00 2024-09-10T18:09:05.903 INFO:teuthology.orchestra.run.smithi099.stdout:(37/115): fuse-2.9.9-16.el9.x86_64.rpm 562 kB/s | 80 kB 00:00 2024-09-10T18:09:06.019 INFO:teuthology.orchestra.run.smithi099.stdout:(38/115): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-10T18:09:06.045 INFO:teuthology.orchestra.run.smithi099.stdout:(39/115): libquadmath-11.5.0-2.el9.x86_64.rpm 954 kB/s | 191 kB 00:00 2024-09-10T18:09:06.078 INFO:teuthology.orchestra.run.smithi099.stdout:(40/115): libgfortran-11.5.0-2.el9.x86_64.rpm 3.0 MB/s | 801 kB 00:00 2024-09-10T18:09:06.085 INFO:teuthology.orchestra.run.smithi133.stdout:(2/115): ceph-fuse-17.2.7-1606.g5be57395.el9.x8 3.1 MB/s | 813 kB 00:00 2024-09-10T18:09:06.103 INFO:teuthology.orchestra.run.smithi099.stdout:(41/115): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-10T18:09:06.137 INFO:teuthology.orchestra.run.smithi099.stdout:(42/115): python3-pycparser-2.20-6.el9.noarch.r 2.2 MB/s | 135 kB 00:00 2024-09-10T18:09:06.163 INFO:teuthology.orchestra.run.smithi099.stdout:(43/115): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-10T18:09:06.194 INFO:teuthology.orchestra.run.smithi133.stdout:(3/115): ceph-base-17.2.7-1606.g5be57395.el9.x8 9.8 MB/s | 5.0 MB 00:00 2024-09-10T18:09:06.196 INFO:teuthology.orchestra.run.smithi099.stdout:(44/115): python3-cryptography-36.0.1-4.el9.x86 6.9 MB/s | 1.2 MB 00:00 2024-09-10T18:09:06.220 INFO:teuthology.orchestra.run.smithi133.stdout:(4/115): ceph-immutable-object-cache-17.2.7-160 1.0 MB/s | 137 kB 00:00 2024-09-10T18:09:06.256 INFO:teuthology.orchestra.run.smithi099.stdout:(45/115): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-10T18:09:06.330 INFO:teuthology.orchestra.run.smithi099.stdout:(46/115): flexiblas-3.0.4-8.el9.x86_64.rpm 196 kB/s | 33 kB 00:00 2024-09-10T18:09:06.355 INFO:teuthology.orchestra.run.smithi099.stdout:(47/115): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-09-10T18:09:06.386 INFO:teuthology.orchestra.run.smithi133.stdout:(5/115): ceph-mds-17.2.7-1606.g5be57395.el9.x86 11 MB/s | 2.1 MB 00:00 2024-09-10T18:09:06.430 INFO:teuthology.orchestra.run.smithi099.stdout:(48/115): librabbitmq-0.11.0-7.el9.x86_64.rpm 604 kB/s | 45 kB 00:00 2024-09-10T18:09:06.456 INFO:teuthology.orchestra.run.smithi099.stdout:(49/115): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-10T18:09:06.539 INFO:teuthology.orchestra.run.smithi099.stdout:(50/115): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-10T18:09:06.570 INFO:teuthology.orchestra.run.smithi133.stdout:(6/115): ceph-mgr-17.2.7-1606.g5be57395.el9.x86 4.0 MB/s | 1.4 MB 00:00 2024-09-10T18:09:06.623 INFO:teuthology.orchestra.run.smithi099.stdout:(51/115): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-10T18:09:06.673 INFO:teuthology.orchestra.run.smithi099.stdout:(52/115): openblas-0.3.26-2.el9.x86_64.rpm 288 kB/s | 38 kB 00:00 2024-09-10T18:09:06.678 INFO:teuthology.orchestra.run.smithi133.stdout:(7/115): ceph-mon-17.2.7-1606.g5be57395.el9.x86 13 MB/s | 3.9 MB 00:00 2024-09-10T18:09:06.749 INFO:teuthology.orchestra.run.smithi099.stdout:(53/115): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-10T18:09:06.941 INFO:teuthology.orchestra.run.smithi099.stdout:(54/115): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.0 MB/s | 3.0 MB 00:00 2024-09-10T18:09:06.979 INFO:teuthology.orchestra.run.smithi133.stdout:(8/115): ceph-common-17.2.7-1606.g5be57395.el9. 17 MB/s | 22 MB 00:01 2024-09-10T18:09:07.024 INFO:teuthology.orchestra.run.smithi099.stdout:(55/115): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-10T18:09:07.029 INFO:teuthology.orchestra.run.smithi133.stdout:(9/115): ceph-selinux-17.2.7-1606.g5be57395.el9 507 kB/s | 25 kB 00:00 2024-09-10T18:09:07.074 INFO:teuthology.orchestra.run.smithi099.stdout:(56/115): python3-iniconfig-1.1.1-7.el9.noarch. 350 kB/s | 17 kB 00:00 2024-09-10T18:09:07.158 INFO:teuthology.orchestra.run.smithi099.stdout:(57/115): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-10T18:09:07.208 INFO:teuthology.orchestra.run.smithi099.stdout:(58/115): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-09-10T18:09:07.258 INFO:teuthology.orchestra.run.smithi099.stdout:(59/115): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-10T18:09:07.342 INFO:teuthology.orchestra.run.smithi099.stdout:(60/115): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-10T18:09:07.476 INFO:teuthology.orchestra.run.smithi099.stdout:(61/115): python3-markupsafe-1.1.1-12.el9.x86_6 261 kB/s | 35 kB 00:00 2024-09-10T18:09:07.542 INFO:teuthology.orchestra.run.smithi099.stdout:(62/115): openblas-openmp-0.3.26-2.el9.x86_64.r 5.4 MB/s | 4.9 MB 00:00 2024-09-10T18:09:07.571 INFO:teuthology.orchestra.run.smithi133.stdout:(10/115): ceph-radosgw-17.2.7-1606.g5be57395.el 16 MB/s | 14 MB 00:00 2024-09-10T18:09:07.626 INFO:teuthology.orchestra.run.smithi099.stdout:(63/115): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-10T18:09:07.629 INFO:teuthology.orchestra.run.smithi133.stdout:(11/115): libcephfs-devel-17.2.7-1606.g5be57395 430 kB/s | 25 kB 00:00 2024-09-10T18:09:07.710 INFO:teuthology.orchestra.run.smithi099.stdout:(64/115): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-10T18:09:07.713 INFO:teuthology.orchestra.run.smithi133.stdout:(12/115): libcephfs2-17.2.7-1606.g5be57395.el9. 7.5 MB/s | 637 kB 00:00 2024-09-10T18:09:07.768 INFO:teuthology.orchestra.run.smithi099.stdout:(65/115): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-10T18:09:07.780 INFO:teuthology.orchestra.run.smithi133.stdout:(13/115): libcephsqlite-17.2.7-1606.g5be57395.e 2.4 MB/s | 163 kB 00:00 2024-09-10T18:09:07.827 INFO:teuthology.orchestra.run.smithi099.stdout:(66/115): python3-pluggy-0.13.1-7.el9.noarch.rp 712 kB/s | 41 kB 00:00 2024-09-10T18:09:07.847 INFO:teuthology.orchestra.run.smithi133.stdout:(14/115): librados-devel-17.2.7-1606.g5be57395. 1.8 MB/s | 126 kB 00:00 2024-09-10T18:09:07.877 INFO:teuthology.orchestra.run.smithi099.stdout:(67/115): python3-prettytable-0.7.2-27.el9.noar 848 kB/s | 42 kB 00:00 2024-09-10T18:09:07.922 INFO:teuthology.orchestra.run.smithi133.stdout:(15/115): libradosstriper1-17.2.7-1606.g5be5739 6.0 MB/s | 458 kB 00:00 2024-09-10T18:09:08.019 INFO:teuthology.orchestra.run.smithi099.stdout:(68/115): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-09-10T18:09:08.044 INFO:teuthology.orchestra.run.smithi099.stdout:(69/115): python3-py-1.10.0-6.el9.noarch.rpm 2.8 MB/s | 477 kB 00:00 2024-09-10T18:09:08.069 INFO:teuthology.orchestra.run.smithi099.stdout:(70/115): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T18:09:08.128 INFO:teuthology.orchestra.run.smithi099.stdout:(71/115): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-09-10T18:09:08.187 INFO:teuthology.orchestra.run.smithi099.stdout:(72/115): python3-pytz-2021.1-5.el9.noarch.rpm 874 kB/s | 51 kB 00:00 2024-09-10T18:09:08.213 INFO:teuthology.orchestra.run.smithi099.stdout:(73/115): python3-pytest-6.2.2-7.el9.noarch.rpm 3.6 MB/s | 519 kB 00:00 2024-09-10T18:09:08.222 INFO:teuthology.orchestra.run.smithi133.stdout:(16/115): librgw2-17.2.7-1606.g5be57395.el9.x86 16 MB/s | 4.9 MB 00:00 2024-09-10T18:09:08.237 INFO:teuthology.orchestra.run.smithi099.stdout:(74/115): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T18:09:08.288 INFO:teuthology.orchestra.run.smithi099.stdout:(75/115): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-09-10T18:09:08.289 INFO:teuthology.orchestra.run.smithi133.stdout:(17/115): python3-ceph-argparse-17.2.7-1606.g5b 678 kB/s | 45 kB 00:00 2024-09-10T18:09:08.356 INFO:teuthology.orchestra.run.smithi133.stdout:(18/115): python3-ceph-common-17.2.7-1606.g5be5 1.8 MB/s | 120 kB 00:00 2024-09-10T18:09:08.371 INFO:teuthology.orchestra.run.smithi099.stdout:(76/115): socat-1.7.4.1-6.el9.x86_64.rpm 3.6 MB/s | 304 kB 00:00 2024-09-10T18:09:08.422 INFO:teuthology.orchestra.run.smithi099.stdout:(77/115): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T18:09:08.423 INFO:teuthology.orchestra.run.smithi133.stdout:(19/115): python3-cephfs-17.2.7-1606.g5be57395. 2.3 MB/s | 156 kB 00:00 2024-09-10T18:09:08.472 INFO:teuthology.orchestra.run.smithi099.stdout:(78/115): gperftools-libs-2.9.1-2.el9.x86_64.rp 6.1 MB/s | 309 kB 00:00 2024-09-10T18:09:08.497 INFO:teuthology.orchestra.run.smithi099.stdout:(79/115): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T18:09:08.522 INFO:teuthology.orchestra.run.smithi099.stdout:(80/115): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T18:09:08.556 INFO:teuthology.orchestra.run.smithi099.stdout:(81/115): python3-asyncssh-2.13.2-5.el9.noarch. 16 MB/s | 548 kB 00:00 2024-09-10T18:09:08.573 INFO:teuthology.orchestra.run.smithi133.stdout:(20/115): ceph-osd-17.2.7-1606.g5be57395.el9.x8 7.7 MB/s | 15 MB 00:02 2024-09-10T18:09:08.581 INFO:teuthology.orchestra.run.smithi099.stdout:(82/115): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-09-10T18:09:08.598 INFO:teuthology.orchestra.run.smithi133.stdout:(21/115): python3-rados-17.2.7-1606.g5be57395.e 1.8 MB/s | 320 kB 00:00 2024-09-10T18:09:08.606 INFO:teuthology.orchestra.run.smithi099.stdout:(83/115): python3-cachetools-4.2.4-1.el9.noarch 1.3 MB/s | 32 kB 00:00 2024-09-10T18:09:08.623 INFO:teuthology.orchestra.run.smithi133.stdout:(22/115): python3-rbd-17.2.7-1606.g5be57395.el9 5.8 MB/s | 295 kB 00:00 2024-09-10T18:09:08.632 INFO:teuthology.orchestra.run.smithi099.stdout:(84/115): python3-certifi-2023.05.07-4.el9.noar 565 kB/s | 14 kB 00:00 2024-09-10T18:09:08.649 INFO:teuthology.orchestra.run.smithi133.stdout:(23/115): python3-rgw-17.2.7-1606.g5be57395.el9 1.9 MB/s | 99 kB 00:00 2024-09-10T18:09:08.657 INFO:teuthology.orchestra.run.smithi099.stdout:(85/115): python3-cheroot-8.6.0-4.el9.noarch.rp 6.7 MB/s | 172 kB 00:00 2024-09-10T18:09:08.674 INFO:teuthology.orchestra.run.smithi133.stdout:(24/115): rbd-fuse-17.2.7-1606.g5be57395.el9.x8 1.6 MB/s | 82 kB 00:00 2024-09-10T18:09:08.691 INFO:teuthology.orchestra.run.smithi099.stdout:(86/115): python3-cherrypy-18.6.1-2.el9.noarch. 11 MB/s | 358 kB 00:00 2024-09-10T18:09:08.741 INFO:teuthology.orchestra.run.smithi133.stdout:(25/115): rbd-nbd-17.2.7-1606.g5be57395.el9.x86 2.4 MB/s | 165 kB 00:00 2024-09-10T18:09:08.758 INFO:teuthology.orchestra.run.smithi099.stdout:(87/115): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.0 MB/s | 5.1 MB 00:01 2024-09-10T18:09:08.791 INFO:teuthology.orchestra.run.smithi099.stdout:(88/115): python3-google-auth-2.34.0-1.el9.noar 2.1 MB/s | 220 kB 00:00 2024-09-10T18:09:08.816 INFO:teuthology.orchestra.run.smithi133.stdout:(26/115): rbd-mirror-17.2.7-1606.g5be57395.el9. 18 MB/s | 2.9 MB 00:00 2024-09-10T18:09:08.817 INFO:teuthology.orchestra.run.smithi099.stdout:(89/115): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-09-10T18:09:08.843 INFO:teuthology.orchestra.run.smithi133.stdout:(27/115): ceph-grafana-dashboards-17.2.7-1606.g 243 kB/s | 24 kB 00:00 2024-09-10T18:09:08.843 INFO:teuthology.orchestra.run.smithi099.stdout:(90/115): python3-jaraco-classes-3.2.1-5.el9.no 366 kB/s | 18 kB 00:00 2024-09-10T18:09:08.867 INFO:teuthology.orchestra.run.smithi099.stdout:(91/115): python3-jaraco-collections-3.0.0-8.el 937 kB/s | 23 kB 00:00 2024-09-10T18:09:08.875 INFO:teuthology.orchestra.run.smithi133.stdout:(28/115): ceph-mgr-cephadm-17.2.7-1606.g5be5739 2.1 MB/s | 124 kB 00:00 2024-09-10T18:09:08.892 INFO:teuthology.orchestra.run.smithi099.stdout:(92/115): python3-jaraco-functools-3.5.0-2.el9. 389 kB/s | 19 kB 00:00 2024-09-10T18:09:08.917 INFO:teuthology.orchestra.run.smithi099.stdout:(93/115): python3-jaraco-text-3.2.0-6.el9.noarc 392 kB/s | 20 kB 00:00 2024-09-10T18:09:08.943 INFO:teuthology.orchestra.run.smithi099.stdout:(94/115): python3-jwt+crypto-2.4.0-1.el9.noarch 181 kB/s | 9.0 kB 00:00 2024-09-10T18:09:08.968 INFO:teuthology.orchestra.run.smithi099.stdout:(95/115): python3-jwt-2.4.0-1.el9.noarch.rpm 816 kB/s | 41 kB 00:00 2024-09-10T18:09:08.993 INFO:teuthology.orchestra.run.smithi099.stdout:(96/115): python3-logutils-0.3.5-21.el9.noarch. 1.8 MB/s | 46 kB 00:00 2024-09-10T18:09:09.020 INFO:teuthology.orchestra.run.smithi099.stdout:(97/115): python3-more-itertools-8.12.0-2.el9.n 3.1 MB/s | 79 kB 00:00 2024-09-10T18:09:09.044 INFO:teuthology.orchestra.run.smithi099.stdout:(98/115): python3-natsort-7.1.1-5.el9.noarch.rp 2.3 MB/s | 58 kB 00:00 2024-09-10T18:09:09.077 INFO:teuthology.orchestra.run.smithi099.stdout:(99/115): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-10T18:09:09.111 INFO:teuthology.orchestra.run.smithi099.stdout:(100/115): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-10T18:09:09.136 INFO:teuthology.orchestra.run.smithi099.stdout:(101/115): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-09-10T18:09:09.161 INFO:teuthology.orchestra.run.smithi099.stdout:(102/115): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-10T18:09:09.186 INFO:teuthology.orchestra.run.smithi099.stdout:(103/115): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-10T18:09:09.212 INFO:teuthology.orchestra.run.smithi099.stdout:(104/115): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T18:09:09.245 INFO:teuthology.orchestra.run.smithi099.stdout:(105/115): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-10T18:09:09.271 INFO:teuthology.orchestra.run.smithi099.stdout:(106/115): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-09-10T18:09:09.296 INFO:teuthology.orchestra.run.smithi099.stdout:(107/115): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-10T18:09:09.321 INFO:teuthology.orchestra.run.smithi099.stdout:(108/115): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T18:09:09.347 INFO:teuthology.orchestra.run.smithi099.stdout:(109/115): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T18:09:09.372 INFO:teuthology.orchestra.run.smithi099.stdout:(110/115): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-10T18:09:09.406 INFO:teuthology.orchestra.run.smithi099.stdout:(111/115): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-10T18:09:09.456 INFO:teuthology.orchestra.run.smithi099.stdout:(112/115): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-10T18:09:09.551 INFO:teuthology.orchestra.run.smithi133.stdout:(29/115): ceph-test-17.2.7-1606.g5be57395.el9.x 19 MB/s | 48 MB 00:02 2024-09-10T18:09:09.609 INFO:teuthology.orchestra.run.smithi133.stdout:(30/115): ceph-mgr-dashboard-17.2.7-1606.g5be57 4.4 MB/s | 3.4 MB 00:00 2024-09-10T18:09:09.634 INFO:teuthology.orchestra.run.smithi133.stdout:(31/115): ceph-mgr-modules-core-17.2.7-1606.g5b 2.8 MB/s | 240 kB 00:00 2024-09-10T18:09:09.673 INFO:teuthology.orchestra.run.smithi099.stdout:(113/115): librados2-17.2.7-1606.g5be57395.el9. 12 MB/s | 3.1 MB 00:00 2024-09-10T18:09:09.718 INFO:teuthology.orchestra.run.smithi133.stdout:(32/115): ceph-mgr-diskprediction-local-17.2.7- 8.8 MB/s | 7.4 MB 00:00 2024-09-10T18:09:09.743 INFO:teuthology.orchestra.run.smithi133.stdout:(33/115): ceph-prometheus-alerts-17.2.7-1606.g5 134 kB/s | 15 kB 00:00 2024-09-10T18:09:09.756 INFO:teuthology.orchestra.run.smithi099.stdout:(114/115): librbd1-17.2.7-1606.g5be57395.el9.x8 9.8 MB/s | 2.9 MB 00:00 2024-09-10T18:09:09.768 INFO:teuthology.orchestra.run.smithi133.stdout:(34/115): ceph-mgr-rook-17.2.7-1606.g5be57395.e 299 kB/s | 47 kB 00:00 2024-09-10T18:09:09.794 INFO:teuthology.orchestra.run.smithi133.stdout:(35/115): cephadm-17.2.7-1606.g5be57395.el9.noa 1.6 MB/s | 83 kB 00:00 2024-09-10T18:09:09.819 INFO:teuthology.orchestra.run.smithi133.stdout:(36/115): ceph-volume-17.2.7-1606.g5be57395.el9 2.6 MB/s | 270 kB 00:00 2024-09-10T18:09:09.936 INFO:teuthology.orchestra.run.smithi133.stdout:(37/115): fuse-2.9.9-16.el9.x86_64.rpm 477 kB/s | 80 kB 00:00 2024-09-10T18:09:09.970 INFO:teuthology.orchestra.run.smithi133.stdout:(38/115): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-10T18:09:10.003 INFO:teuthology.orchestra.run.smithi133.stdout:(39/115): libgfortran-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 801 kB 00:00 2024-09-10T18:09:10.029 INFO:teuthology.orchestra.run.smithi133.stdout:(40/115): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-10T18:09:10.079 INFO:teuthology.orchestra.run.smithi133.stdout:(41/115): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-10T18:09:10.104 INFO:teuthology.orchestra.run.smithi133.stdout:(42/115): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-09-10T18:09:10.138 INFO:teuthology.orchestra.run.smithi133.stdout:(43/115): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-10T18:09:10.163 INFO:teuthology.orchestra.run.smithi133.stdout:(44/115): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-10T18:09:10.205 INFO:teuthology.orchestra.run.smithi133.stdout:(45/115): python3-urllib3-1.26.5-6.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-09-10T18:09:10.280 INFO:teuthology.orchestra.run.smithi133.stdout:(46/115): flexiblas-3.0.4-8.el9.x86_64.rpm 230 kB/s | 33 kB 00:00 2024-09-10T18:09:10.305 INFO:teuthology.orchestra.run.smithi133.stdout:(47/115): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-09-10T18:09:10.380 INFO:teuthology.orchestra.run.smithi133.stdout:(48/115): librabbitmq-0.11.0-7.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-09-10T18:09:10.406 INFO:teuthology.orchestra.run.smithi133.stdout:(49/115): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-10T18:09:10.514 INFO:teuthology.orchestra.run.smithi133.stdout:(50/115): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-10T18:09:10.565 INFO:teuthology.orchestra.run.smithi133.stdout:(51/115): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-10T18:09:10.598 INFO:teuthology.orchestra.run.smithi133.stdout:(52/115): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-09-10T18:09:10.832 INFO:teuthology.orchestra.run.smithi133.stdout:(53/115): python3-attrs-20.3.0-7.el9.noarch.rpm 375 kB/s | 87 kB 00:00 2024-09-10T18:09:10.965 INFO:teuthology.orchestra.run.smithi099.stdout:(115/115): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.9 MB/s | 16 MB 00:02 2024-09-10T18:09:10.969 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:09:10.969 INFO:teuthology.orchestra.run.smithi099.stdout:Total 16 MB/s | 191 MB 00:11 2024-09-10T18:09:10.990 INFO:teuthology.orchestra.run.smithi133.stdout:(54/115): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.6 MB/s | 3.0 MB 00:00 2024-09-10T18:09:11.074 INFO:teuthology.orchestra.run.smithi133.stdout:(55/115): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-10T18:09:11.133 INFO:teuthology.orchestra.run.smithi133.stdout:(56/115): python3-iniconfig-1.1.1-7.el9.noarch. 299 kB/s | 17 kB 00:00 2024-09-10T18:09:11.208 INFO:teuthology.orchestra.run.smithi133.stdout:(57/115): python3-jinja2-2.11.3-6.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-09-10T18:09:11.258 INFO:teuthology.orchestra.run.smithi133.stdout:(58/115): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-09-10T18:09:11.308 INFO:teuthology.orchestra.run.smithi133.stdout:(59/115): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-10T18:09:11.359 INFO:teuthology.orchestra.run.smithi133.stdout:(60/115): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-10T18:09:11.492 INFO:teuthology.orchestra.run.smithi133.stdout:(61/115): openblas-openmp-0.3.26-2.el9.x86_64.r 5.3 MB/s | 4.9 MB 00:00 2024-09-10T18:09:11.518 INFO:teuthology.orchestra.run.smithi133.stdout:(62/115): python3-markupsafe-1.1.1-12.el9.x86_6 219 kB/s | 35 kB 00:00 2024-09-10T18:09:11.651 INFO:teuthology.orchestra.run.smithi133.stdout:(63/115): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-09-10T18:09:11.698 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-10T18:09:11.735 INFO:teuthology.orchestra.run.smithi133.stdout:(64/115): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-10T18:09:11.793 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-10T18:09:11.793 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-10T18:09:11.794 INFO:teuthology.orchestra.run.smithi133.stdout:(65/115): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-10T18:09:11.844 INFO:teuthology.orchestra.run.smithi133.stdout:(66/115): python3-pluggy-0.13.1-7.el9.noarch.rp 824 kB/s | 41 kB 00:00 2024-09-10T18:09:11.894 INFO:teuthology.orchestra.run.smithi133.stdout:(67/115): python3-prettytable-0.7.2-27.el9.noar 847 kB/s | 42 kB 00:00 2024-09-10T18:09:12.045 INFO:teuthology.orchestra.run.smithi133.stdout:(68/115): python3-babel-2.9.1-2.el9.noarch.rpm 4.9 MB/s | 6.0 MB 00:01 2024-09-10T18:09:12.070 INFO:teuthology.orchestra.run.smithi133.stdout:(69/115): python3-py-1.10.0-6.el9.noarch.rpm 2.7 MB/s | 477 kB 00:00 2024-09-10T18:09:12.095 INFO:teuthology.orchestra.run.smithi133.stdout:(70/115): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T18:09:12.162 INFO:teuthology.orchestra.run.smithi133.stdout:(71/115): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-09-10T18:09:12.220 INFO:teuthology.orchestra.run.smithi133.stdout:(72/115): python3-pytest-6.2.2-7.el9.noarch.rpm 4.1 MB/s | 519 kB 00:00 2024-09-10T18:09:12.245 INFO:teuthology.orchestra.run.smithi133.stdout:(73/115): python3-pytz-2021.1-5.el9.noarch.rpm 612 kB/s | 51 kB 00:00 2024-09-10T18:09:12.272 INFO:teuthology.orchestra.run.smithi133.stdout:(74/115): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T18:09:12.363 INFO:teuthology.orchestra.run.smithi133.stdout:(75/115): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.9 MB/s | 5.1 MB 00:00 2024-09-10T18:09:12.388 INFO:teuthology.orchestra.run.smithi133.stdout:(76/115): python3-toml-0.10.2-6.el9.noarch.rpm 356 kB/s | 42 kB 00:00 2024-09-10T18:09:12.438 INFO:teuthology.orchestra.run.smithi133.stdout:(77/115): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T18:09:12.463 INFO:teuthology.orchestra.run.smithi133.stdout:(78/115): socat-1.7.4.1-6.el9.x86_64.rpm 3.0 MB/s | 304 kB 00:00 2024-09-10T18:09:12.514 INFO:teuthology.orchestra.run.smithi133.stdout:(79/115): gperftools-libs-2.9.1-2.el9.x86_64.rp 4.0 MB/s | 309 kB 00:00 2024-09-10T18:09:12.531 INFO:teuthology.orchestra.run.smithi133.stdout:(80/115): liboath-2.6.7-2.el9.x86_64.rpm 726 kB/s | 49 kB 00:00 2024-09-10T18:09:12.556 INFO:teuthology.orchestra.run.smithi133.stdout:(81/115): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-10T18:09:12.573 INFO:teuthology.orchestra.run.smithi133.stdout:(82/115): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 2.6 MB/s | 43 kB 00:00 2024-09-10T18:09:12.590 INFO:teuthology.orchestra.run.smithi133.stdout:(83/115): python3-cachetools-4.2.4-1.el9.noarch 1.9 MB/s | 32 kB 00:00 2024-09-10T18:09:12.607 INFO:teuthology.orchestra.run.smithi133.stdout:(84/115): python3-certifi-2023.05.07-4.el9.noar 854 kB/s | 14 kB 00:00 2024-09-10T18:09:12.640 INFO:teuthology.orchestra.run.smithi133.stdout:(85/115): python3-asyncssh-2.13.2-5.el9.noarch. 4.9 MB/s | 548 kB 00:00 2024-09-10T18:09:12.665 INFO:teuthology.orchestra.run.smithi133.stdout:(86/115): python3-cheroot-8.6.0-4.el9.noarch.rp 2.9 MB/s | 172 kB 00:00 2024-09-10T18:09:12.699 INFO:teuthology.orchestra.run.smithi133.stdout:(87/115): python3-cherrypy-18.6.1-2.el9.noarch. 6.0 MB/s | 358 kB 00:00 2024-09-10T18:09:12.725 INFO:teuthology.orchestra.run.smithi133.stdout:(88/115): python3-google-auth-2.34.0-1.el9.noar 3.7 MB/s | 220 kB 00:00 2024-09-10T18:09:12.758 INFO:teuthology.orchestra.run.smithi133.stdout:(89/115): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-09-10T18:09:12.783 INFO:teuthology.orchestra.run.smithi133.stdout:(90/115): python3-jaraco-classes-3.2.1-5.el9.no 303 kB/s | 18 kB 00:00 2024-09-10T18:09:12.809 INFO:teuthology.orchestra.run.smithi133.stdout:(91/115): python3-jaraco-collections-3.0.0-8.el 460 kB/s | 23 kB 00:00 2024-09-10T18:09:12.834 INFO:teuthology.orchestra.run.smithi133.stdout:(92/115): python3-jaraco-functools-3.5.0-2.el9. 388 kB/s | 19 kB 00:00 2024-09-10T18:09:12.859 INFO:teuthology.orchestra.run.smithi133.stdout:(93/115): python3-jaraco-text-3.2.0-6.el9.noarc 393 kB/s | 20 kB 00:00 2024-09-10T18:09:12.884 INFO:teuthology.orchestra.run.smithi133.stdout:(94/115): python3-jwt+crypto-2.4.0-1.el9.noarch 180 kB/s | 9.0 kB 00:00 2024-09-10T18:09:12.912 INFO:teuthology.orchestra.run.smithi133.stdout:(95/115): python3-jwt-2.4.0-1.el9.noarch.rpm 814 kB/s | 41 kB 00:00 2024-09-10T18:09:12.935 INFO:teuthology.orchestra.run.smithi133.stdout:(96/115): python3-logutils-0.3.5-21.el9.noarch. 1.8 MB/s | 46 kB 00:00 2024-09-10T18:09:12.960 INFO:teuthology.orchestra.run.smithi133.stdout:(97/115): python3-more-itertools-8.12.0-2.el9.n 3.1 MB/s | 79 kB 00:00 2024-09-10T18:09:12.985 INFO:teuthology.orchestra.run.smithi133.stdout:(98/115): python3-natsort-7.1.1-5.el9.noarch.rp 2.3 MB/s | 58 kB 00:00 2024-09-10T18:09:13.019 INFO:teuthology.orchestra.run.smithi133.stdout:(99/115): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-10T18:09:13.053 INFO:teuthology.orchestra.run.smithi133.stdout:(100/115): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-10T18:09:13.078 INFO:teuthology.orchestra.run.smithi133.stdout:(101/115): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-10T18:09:13.104 INFO:teuthology.orchestra.run.smithi133.stdout:(102/115): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-10T18:09:13.129 INFO:teuthology.orchestra.run.smithi133.stdout:(103/115): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-10T18:09:13.179 INFO:teuthology.orchestra.run.smithi133.stdout:(104/115): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-10T18:09:13.213 INFO:teuthology.orchestra.run.smithi133.stdout:(105/115): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-10T18:09:13.238 INFO:teuthology.orchestra.run.smithi133.stdout:(106/115): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-09-10T18:09:13.271 INFO:teuthology.orchestra.run.smithi133.stdout:(107/115): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-10T18:09:13.296 INFO:teuthology.orchestra.run.smithi133.stdout:(108/115): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T18:09:13.322 INFO:teuthology.orchestra.run.smithi133.stdout:(109/115): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-10T18:09:13.347 INFO:teuthology.orchestra.run.smithi133.stdout:(110/115): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-10T18:09:13.381 INFO:teuthology.orchestra.run.smithi133.stdout:(111/115): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-10T18:09:13.423 INFO:teuthology.orchestra.run.smithi133.stdout:(112/115): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-10T18:09:13.446 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-10T18:09:13.447 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-10T18:09:13.648 INFO:teuthology.orchestra.run.smithi133.stdout:(113/115): librbd1-17.2.7-1606.g5be57395.el9.x8 13 MB/s | 2.9 MB 00:00 2024-09-10T18:09:13.715 INFO:teuthology.orchestra.run.smithi133.stdout:(114/115): librados2-17.2.7-1606.g5be57395.el9. 9.2 MB/s | 3.1 MB 00:00 2024-09-10T18:09:15.657 INFO:teuthology.orchestra.run.smithi133.stdout:(115/115): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.8 MB/s | 16 MB 00:03 2024-09-10T18:09:15.662 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:09:15.662 INFO:teuthology.orchestra.run.smithi133.stdout:Total 19 MB/s | 191 MB 00:10 2024-09-10T18:09:16.425 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-09-10T18:09:16.521 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-09-10T18:09:16.521 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-09-10T18:09:18.067 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-10T18:09:18.125 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/117 2024-09-10T18:09:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/117 2024-09-10T18:09:18.174 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-09-10T18:09:18.175 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-09-10T18:09:18.215 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/117 2024-09-10T18:09:18.520 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/117 2024-09-10T18:09:18.617 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 5/117 2024-09-10T18:09:18.731 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 5/117 2024-09-10T18:09:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 6/117 2024-09-10T18:09:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 6/117 2024-09-10T18:09:18.902 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 7/117 2024-09-10T18:09:18.940 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/117 2024-09-10T18:09:19.388 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/117 2024-09-10T18:09:19.413 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 10/117 2024-09-10T18:09:19.493 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 10/117 2024-09-10T18:09:19.536 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-packaging-20.9-5.el9.noarch 11/117 2024-09-10T18:09:19.564 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 12/117 2024-09-10T18:09:19.648 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 12/117 2024-09-10T18:09:19.673 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86 13/117 2024-09-10T18:09:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86 13/117 2024-09-10T18:09:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 14/117 2024-09-10T18:09:19.877 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.7-2.el9.x86_64 15/117 2024-09-10T18:09:19.907 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/117 2024-09-10T18:09:19.947 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/117 2024-09-10T18:09:20.012 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 18/117 2024-09-10T18:09:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 19/117 2024-09-10T18:09:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 20/117 2024-09-10T18:09:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 21/117 2024-09-10T18:09:20.363 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/117 2024-09-10T18:09:20.399 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/117 2024-09-10T18:09:20.437 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 24/117 2024-09-10T18:09:20.512 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 25/117 2024-09-10T18:09:20.547 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9. 26/117 2024-09-10T18:09:20.590 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el 27/117 2024-09-10T18:09:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_6 28/117 2024-09-10T18:09:20.823 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/117 2024-09-10T18:09:20.876 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 30/117 2024-09-10T18:09:21.415 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el9.noarch 31/117 2024-09-10T18:09:21.497 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 32/117 2024-09-10T18:09:21.543 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 33/117 2024-09-10T18:09:21.577 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 34/117 2024-09-10T18:09:21.837 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 35/117 2024-09-10T18:09:21.876 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 36/117 2024-09-10T18:09:21.913 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 37/117 2024-09-10T18:09:21.950 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_6 38/117 2024-09-10T18:09:21.980 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/117 2024-09-10T18:09:22.017 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 40/117 2024-09-10T18:09:22.072 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/117 2024-09-10T18:09:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/117 2024-09-10T18:09:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/117 2024-09-10T18:09:22.220 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/117 2024-09-10T18:09:22.268 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/117 2024-09-10T18:09:22.312 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/117 2024-09-10T18:09:22.354 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/117 2024-09-10T18:09:22.493 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/117 2024-09-10T18:09:22.531 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/117 2024-09-10T18:09:22.572 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/117 2024-09-10T18:09:22.676 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/117 2024-09-10T18:09:22.714 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/117 2024-09-10T18:09:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/117 2024-09-10T18:09:22.774 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-09-10T18:09:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 54/117 2024-09-10T18:09:22.836 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 55/117 2024-09-10T18:09:22.850 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/117 2024-09-10T18:09:22.909 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/117 2024-09-10T18:09:22.961 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 56/117 2024-09-10T18:09:22.976 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/117 2024-09-10T18:09:22.995 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-py-1.10.0-6.el9.noarch 57/117 2024-09-10T18:09:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 58/117 2024-09-10T18:09:23.133 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 59/117 2024-09-10T18:09:23.187 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 60/117 2024-09-10T18:09:23.279 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/117 2024-09-10T18:09:23.376 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 5/117 2024-09-10T18:09:23.507 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 5/117 2024-09-10T18:09:23.552 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 6/117 2024-09-10T18:09:23.655 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 6/117 2024-09-10T18:09:23.695 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 7/117 2024-09-10T18:09:23.724 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/117 2024-09-10T18:09:23.930 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 61/117 2024-09-10T18:09:23.976 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 62/117 2024-09-10T18:09:24.010 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 63/117 2024-09-10T18:09:24.043 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 64/117 2024-09-10T18:09:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 65/117 2024-09-10T18:09:24.162 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/117 2024-09-10T18:09:24.188 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 10/117 2024-09-10T18:09:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 66/117 2024-09-10T18:09:24.236 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 67/117 2024-09-10T18:09:24.252 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 10/117 2024-09-10T18:09:24.295 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-packaging-20.9-5.el9.noarch 11/117 2024-09-10T18:09:24.323 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 12/117 2024-09-10T18:09:24.419 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.26-2.el9.x86_64 68/117 2024-09-10T18:09:24.423 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 12/117 2024-09-10T18:09:24.449 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86 13/117 2024-09-10T18:09:24.451 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 69/117 2024-09-10T18:09:24.529 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 70/117 2024-09-10T18:09:24.576 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86 13/117 2024-09-10T18:09:24.612 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 14/117 2024-09-10T18:09:24.643 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.7-2.el9.x86_64 15/117 2024-09-10T18:09:24.673 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/117 2024-09-10T18:09:24.706 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/117 2024-09-10T18:09:24.771 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 18/117 2024-09-10T18:09:24.805 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 19/117 2024-09-10T18:09:24.832 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 20/117 2024-09-10T18:09:24.901 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 21/117 2024-09-10T18:09:24.955 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/117 2024-09-10T18:09:24.989 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/117 2024-09-10T18:09:25.029 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 24/117 2024-09-10T18:09:25.112 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 25/117 2024-09-10T18:09:25.148 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9. 26/117 2024-09-10T18:09:25.189 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 71/117 2024-09-10T18:09:25.196 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el 27/117 2024-09-10T18:09:25.274 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_6 28/117 2024-09-10T18:09:25.319 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 72/117 2024-09-10T18:09:25.417 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/117 2024-09-10T18:09:25.469 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 30/117 2024-09-10T18:09:25.999 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.9-2.el9.noarch 31/117 2024-09-10T18:09:26.081 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 32/117 2024-09-10T18:09:26.118 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 33/117 2024-09-10T18:09:26.153 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 34/117 2024-09-10T18:09:26.187 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 35/117 2024-09-10T18:09:26.226 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 36/117 2024-09-10T18:09:26.262 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 37/117 2024-09-10T18:09:26.300 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_6 38/117 2024-09-10T18:09:26.330 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/117 2024-09-10T18:09:26.358 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 40/117 2024-09-10T18:09:26.414 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/117 2024-09-10T18:09:26.487 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/117 2024-09-10T18:09:26.527 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/117 2024-09-10T18:09:26.563 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/117 2024-09-10T18:09:26.599 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/117 2024-09-10T18:09:26.610 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/117 2024-09-10T18:09:26.655 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/117 2024-09-10T18:09:26.684 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 74/117 2024-09-10T18:09:26.697 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/117 2024-09-10T18:09:26.834 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/117 2024-09-10T18:09:26.890 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/117 2024-09-10T18:09:26.931 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/117 2024-09-10T18:09:27.035 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/117 2024-09-10T18:09:27.082 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/117 2024-09-10T18:09:27.138 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 75/117 2024-09-10T18:09:27.147 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/117 2024-09-10T18:09:27.179 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 76/117 2024-09-10T18:09:27.184 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 54/117 2024-09-10T18:09:27.237 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 55/117 2024-09-10T18:09:27.252 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 76/117 2024-09-10T18:09:27.289 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 77/117 2024-09-10T18:09:27.363 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 56/117 2024-09-10T18:09:27.403 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-py-1.10.0-6.el9.noarch 57/117 2024-09-10T18:09:27.442 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 58/117 2024-09-10T18:09:27.541 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 59/117 2024-09-10T18:09:27.597 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 60/117 2024-09-10T18:09:28.329 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 61/117 2024-09-10T18:09:28.517 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 62/117 2024-09-10T18:09:28.603 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 63/117 2024-09-10T18:09:28.637 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 64/117 2024-09-10T18:09:28.740 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 65/117 2024-09-10T18:09:28.802 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 66/117 2024-09-10T18:09:28.837 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 67/117 2024-09-10T18:09:29.025 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.26-2.el9.x86_64 68/117 2024-09-10T18:09:29.060 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 69/117 2024-09-10T18:09:29.141 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 70/117 2024-09-10T18:09:29.760 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 78/117 2024-09-10T18:09:29.794 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 78/117 2024-09-10T18:09:29.813 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 71/117 2024-09-10T18:09:29.940 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 72/117 2024-09-10T18:09:30.322 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 78/117 2024-09-10T18:09:30.364 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 79/117 2024-09-10T18:09:30.461 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 79/117 2024-09-10T18:09:30.462 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T18:09:30.462 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T18:09:30.462 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:09:30.490 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 80/117 2024-09-10T18:09:31.220 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/117 2024-09-10T18:09:31.303 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 74/117 2024-09-10T18:09:31.594 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 75/117 2024-09-10T18:09:31.640 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 76/117 2024-09-10T18:09:31.730 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 76/117 2024-09-10T18:09:31.758 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 77/117 2024-09-10T18:09:34.148 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 78/117 2024-09-10T18:09:34.204 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 78/117 2024-09-10T18:09:34.724 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 78/117 2024-09-10T18:09:34.757 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 79/117 2024-09-10T18:09:34.854 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 79/117 2024-09-10T18:09:34.855 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T18:09:34.855 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T18:09:34.855 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:09:34.892 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 80/117 2024-09-10T18:09:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 80/117 2024-09-10T18:09:42.041 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2024-09-10T18:09:42.041 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2024-09-10T18:09:42.041 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2024-09-10T18:09:42.041 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2024-09-10T18:09:42.041 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2024-09-10T18:09:42.041 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /root 2024-09-10T18:09:42.041 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2024-09-10T18:09:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:09:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 81/117 2024-09-10T18:09:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 81/117 2024-09-10T18:09:42.215 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:42.215 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T18:09:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:09:43.567 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 82/117 2024-09-10T18:09:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 82/117 2024-09-10T18:09:43.596 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:43.597 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T18:09:43.597 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:09:43.597 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:09:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:09:43.801 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 83/117 2024-09-10T18:09:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 83/117 2024-09-10T18:09:43.825 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:43.825 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T18:09:43.825 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:09:43.825 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:09:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:09:44.203 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 84/117 2024-09-10T18:09:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 84/117 2024-09-10T18:09:44.228 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:44.228 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T18:09:44.228 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:09:44.228 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:09:44.228 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:09:44.299 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ply-3.11-14.el9.noarch 85/117 2024-09-10T18:09:44.354 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 86/117 2024-09-10T18:09:44.540 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 87/117 2024-09-10T18:09:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 88/117 2024-09-10T18:09:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 89/117 2024-09-10T18:09:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 90/117 2024-09-10T18:09:44.911 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 91/117 2024-09-10T18:09:44.966 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 92/117 2024-09-10T18:09:44.995 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 93/117 2024-09-10T18:09:45.034 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 94/117 2024-09-10T18:09:45.069 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 95/117 2024-09-10T18:09:45.097 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.9-16.el9.x86_64 96/117 2024-09-10T18:09:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 97/117 2024-09-10T18:09:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 97/117 2024-09-10T18:09:45.565 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.e 98/117 2024-09-10T18:09:45.623 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395. 99/117 2024-09-10T18:09:45.651 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noa 100/117 2024-09-10T18:09:46.626 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 80/117 2024-09-10T18:09:46.626 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2024-09-10T18:09:46.626 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2024-09-10T18:09:46.626 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2024-09-10T18:09:46.627 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2024-09-10T18:09:46.627 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2024-09-10T18:09:46.627 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /root 2024-09-10T18:09:46.627 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2024-09-10T18:09:46.627 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:09:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noa 100/117 2024-09-10T18:09:46.670 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.n 101/117 2024-09-10T18:09:46.800 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 81/117 2024-09-10T18:09:46.816 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 81/117 2024-09-10T18:09:46.816 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:46.816 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T18:09:46.816 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:09:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.n 101/117 2024-09-10T18:09:47.653 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be 102/117 2024-09-10T18:09:47.755 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be 102/117 2024-09-10T18:09:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el 103/117 2024-09-10T18:09:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 104/117 2024-09-10T18:09:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 104/117 2024-09-10T18:09:47.963 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:47.963 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T18:09:47.963 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:09:47.963 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:09:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:09:48.005 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 105/117 2024-09-10T18:09:48.017 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 105/117 2024-09-10T18:09:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 106/117 2024-09-10T18:09:48.162 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 82/117 2024-09-10T18:09:48.189 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 82/117 2024-09-10T18:09:48.189 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:48.189 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T18:09:48.189 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:09:48.189 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:09:48.189 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:09:48.395 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 83/117 2024-09-10T18:09:48.419 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 83/117 2024-09-10T18:09:48.419 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:48.419 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T18:09:48.419 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:09:48.419 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:09:48.419 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:09:48.796 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 84/117 2024-09-10T18:09:48.819 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 84/117 2024-09-10T18:09:48.820 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:48.820 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T18:09:48.820 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:09:48.820 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:09:48.820 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:09:48.881 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ply-3.11-14.el9.noarch 85/117 2024-09-10T18:09:48.936 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 86/117 2024-09-10T18:09:49.119 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 87/117 2024-09-10T18:09:49.170 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 88/117 2024-09-10T18:09:49.247 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 89/117 2024-09-10T18:09:49.335 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 90/117 2024-09-10T18:09:49.351 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 107/117 2024-09-10T18:09:49.382 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 108/117 2024-09-10T18:09:49.460 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 108/117 2024-09-10T18:09:49.460 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:49.460 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T18:09:49.460 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:09:49.461 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:09:49.461 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:09:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 91/117 2024-09-10T18:09:49.500 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57 109/117 2024-09-10T18:09:49.523 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57 109/117 2024-09-10T18:09:49.523 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:49.523 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T18:09:49.523 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:09:49.524 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 92/117 2024-09-10T18:09:49.554 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 93/117 2024-09-10T18:09:49.613 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 94/117 2024-09-10T18:09:49.653 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 95/117 2024-09-10T18:09:49.681 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.9-16.el9.x86_64 96/117 2024-09-10T18:09:49.809 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 110/117 2024-09-10T18:09:49.832 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 110/117 2024-09-10T18:09:49.832 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:49.833 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T18:09:49.833 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:09:49.833 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:09:49.833 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:09:50.114 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 97/117 2024-09-10T18:09:50.146 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 97/117 2024-09-10T18:09:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.e 98/117 2024-09-10T18:09:50.247 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395. 99/117 2024-09-10T18:09:50.285 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noa 100/117 2024-09-10T18:09:51.260 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noa 100/117 2024-09-10T18:09:51.297 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.n 101/117 2024-09-10T18:09:52.257 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.n 101/117 2024-09-10T18:09:52.281 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be 102/117 2024-09-10T18:09:52.383 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be 102/117 2024-09-10T18:09:52.535 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el 103/117 2024-09-10T18:09:52.566 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 104/117 2024-09-10T18:09:52.591 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 104/117 2024-09-10T18:09:52.591 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:52.591 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T18:09:52.591 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:09:52.591 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:09:52.591 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:09:52.634 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 105/117 2024-09-10T18:09:52.646 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 105/117 2024-09-10T18:09:52.737 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 106/117 2024-09-10T18:09:53.986 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 107/117 2024-09-10T18:09:54.027 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 108/117 2024-09-10T18:09:54.114 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 108/117 2024-09-10T18:09:54.114 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:54.114 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T18:09:54.114 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:09:54.114 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:09:54.114 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:09:54.161 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57 109/117 2024-09-10T18:09:54.184 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57 109/117 2024-09-10T18:09:54.184 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:54.185 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T18:09:54.185 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:09:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 111/117 2024-09-10T18:09:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 112/117 2024-09-10T18:09:54.472 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 113/117 2024-09-10T18:09:54.479 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 110/117 2024-09-10T18:09:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 110/117 2024-09-10T18:09:54.503 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:09:54.503 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T18:09:54.503 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:09:54.503 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:09:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:09:54.518 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytest-6.2.2-7.el9.noarch 114/117 2024-09-10T18:09:54.533 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_ 115/117 2024-09-10T18:09:54.533 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 116/117 2024-09-10T18:09:54.616 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 116/117 2024-09-10T18:09:54.617 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 117/117 2024-09-10T18:09:58.898 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 111/117 2024-09-10T18:09:58.939 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 112/117 2024-09-10T18:09:59.059 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 113/117 2024-09-10T18:09:59.097 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytest-6.2.2-7.el9.noarch 114/117 2024-09-10T18:09:59.120 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_ 115/117 2024-09-10T18:09:59.120 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 116/117 2024-09-10T18:09:59.212 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 116/117 2024-09-10T18:09:59.212 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 117/117 2024-09-10T18:09:59.887 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 117/117 2024-09-10T18:09:59.887 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/117 2024-09-10T18:09:59.887 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2/117 2024-09-10T18:09:59.887 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 3/117 2024-09-10T18:09:59.887 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 4/117 2024-09-10T18:09:59.887 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57 5/117 2024-09-10T18:09:59.887 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 6/117 2024-09-10T18:09:59.888 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 7/117 2024-09-10T18:09:59.888 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 8/117 2024-09-10T18:09:59.888 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 9/117 2024-09-10T18:09:59.888 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 10/117 2024-09-10T18:09:59.888 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 11/117 2024-09-10T18:09:59.888 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 12/117 2024-09-10T18:09:59.888 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_ 13/117 2024-09-10T18:09:59.888 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 14/117 2024-09-10T18:09:59.888 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 15/117 2024-09-10T18:09:59.888 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_6 16/117 2024-09-10T18:09:59.888 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86 17/117 2024-09-10T18:09:59.888 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 18/117 2024-09-10T18:09:59.889 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el 19/117 2024-09-10T18:09:59.889 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9. 20/117 2024-09-10T18:09:59.889 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_6 21/117 2024-09-10T18:09:59.889 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 22/117 2024-09-10T18:09:59.889 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 23/117 2024-09-10T18:09:59.889 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/117 2024-09-10T18:09:59.889 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 25/117 2024-09-10T18:09:59.889 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 26/117 2024-09-10T18:09:59.889 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 27/117 2024-09-10T18:09:59.897 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395. 28/117 2024-09-10T18:09:59.897 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noa 29/117 2024-09-10T18:09:59.897 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.n 30/117 2024-09-10T18:09:59.897 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be 31/117 2024-09-10T18:09:59.897 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el 32/117 2024-09-10T18:09:59.897 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 33/117 2024-09-10T18:09:59.898 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.e 34/117 2024-09-10T18:09:59.898 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 35/117 2024-09-10T18:09:59.898 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 36/117 2024-09-10T18:09:59.898 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 37/117 2024-09-10T18:09:59.898 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 38/117 2024-09-10T18:09:59.898 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 39/117 2024-09-10T18:09:59.898 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/117 2024-09-10T18:09:59.898 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/117 2024-09-10T18:09:59.898 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/117 2024-09-10T18:09:59.898 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/117 2024-09-10T18:09:59.898 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/117 2024-09-10T18:09:59.898 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 45/117 2024-09-10T18:09:59.899 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 46/117 2024-09-10T18:09:59.899 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 47/117 2024-09-10T18:09:59.899 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 48/117 2024-09-10T18:09:59.899 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/117 2024-09-10T18:09:59.899 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/117 2024-09-10T18:09:59.899 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/117 2024-09-10T18:09:59.899 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/117 2024-09-10T18:09:59.899 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 53/117 2024-09-10T18:09:59.899 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 54/117 2024-09-10T18:09:59.899 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 55/117 2024-09-10T18:09:59.899 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 56/117 2024-09-10T18:09:59.899 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 57/117 2024-09-10T18:09:59.900 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 58/117 2024-09-10T18:09:59.900 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 59/117 2024-09-10T18:09:59.900 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 60/117 2024-09-10T18:09:59.900 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 61/117 2024-09-10T18:09:59.900 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 62/117 2024-09-10T18:09:59.900 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 63/117 2024-09-10T18:09:59.900 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 64/117 2024-09-10T18:09:59.900 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 65/117 2024-09-10T18:09:59.900 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 66/117 2024-09-10T18:09:59.900 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 67/117 2024-09-10T18:09:59.900 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 68/117 2024-09-10T18:09:59.900 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/117 2024-09-10T18:09:59.901 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 70/117 2024-09-10T18:09:59.901 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/117 2024-09-10T18:09:59.901 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/117 2024-09-10T18:09:59.901 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytest-6.2.2-7.el9.noarch 73/117 2024-09-10T18:09:59.901 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 74/117 2024-09-10T18:09:59.901 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 75/117 2024-09-10T18:09:59.901 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 76/117 2024-09-10T18:09:59.901 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 77/117 2024-09-10T18:09:59.901 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 78/117 2024-09-10T18:09:59.901 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 79/117 2024-09-10T18:09:59.901 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 80/117 2024-09-10T18:09:59.902 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 81/117 2024-09-10T18:09:59.902 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 82/117 2024-09-10T18:09:59.902 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 83/117 2024-09-10T18:09:59.902 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 84/117 2024-09-10T18:09:59.902 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 85/117 2024-09-10T18:09:59.902 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 86/117 2024-09-10T18:09:59.902 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 87/117 2024-09-10T18:09:59.902 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 88/117 2024-09-10T18:09:59.902 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 89/117 2024-09-10T18:09:59.902 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 90/117 2024-09-10T18:09:59.902 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 91/117 2024-09-10T18:09:59.902 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 92/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 93/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 94/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 95/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 96/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 97/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 98/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 99/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 100/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 101/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 102/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 103/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 104/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 105/117 2024-09-10T18:09:59.903 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 106/117 2024-09-10T18:09:59.904 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 107/117 2024-09-10T18:09:59.904 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 108/117 2024-09-10T18:09:59.904 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 109/117 2024-09-10T18:09:59.904 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 110/117 2024-09-10T18:09:59.904 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 111/117 2024-09-10T18:09:59.904 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 112/117 2024-09-10T18:09:59.904 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 113/117 2024-09-10T18:09:59.904 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 114/117 2024-09-10T18:09:59.904 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 115/117 2024-09-10T18:09:59.904 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 116/117 2024-09-10T18:10:02.124 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 117/117 2024-09-10T18:10:02.124 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:10:02.124 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2024-09-10T18:10:02.124 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.124 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.124 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-10T18:10:02.124 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.124 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.125 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.125 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.125 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:02.125 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.125 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.125 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.125 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:02.125 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:02.125 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:02.125 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:02.125 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:02.125 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.126 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.126 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:02.126 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.126 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.126 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.126 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:02.126 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:02.126 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T18:10:02.126 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T18:10:02.126 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T18:10:02.126 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T18:10:02.126 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T18:10:02.127 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.127 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.127 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.127 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T18:10:02.127 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T18:10:02.127 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T18:10:02.127 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T18:10:02.127 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T18:10:02.127 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.127 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.127 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T18:10:02.127 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.127 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T18:10:02.128 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T18:10:02.128 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T18:10:02.128 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T18:10:02.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T18:10:02.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T18:10:02.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T18:10:02.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T18:10:02.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T18:10:02.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T18:10:02.128 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T18:10:02.129 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T18:10:02.129 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T18:10:02.129 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T18:10:02.129 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T18:10:02.129 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T18:10:02.129 INFO:teuthology.orchestra.run.smithi099.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2024-09-10T18:10:02.129 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T18:10:02.129 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T18:10:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T18:10:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T18:10:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T18:10:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T18:10:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T18:10:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T18:10:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T18:10:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T18:10:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T18:10:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T18:10:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T18:10:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T18:10:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T18:10:02.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T18:10:02.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T18:10:02.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T18:10:02.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T18:10:02.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T18:10:02.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T18:10:02.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pluggy-0.13.1-7.el9.noarch 2024-09-10T18:10:02.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T18:10:02.131 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T18:10:02.132 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T18:10:02.132 INFO:teuthology.orchestra.run.smithi099.stdout: python3-py-1.10.0-6.el9.noarch 2024-09-10T18:10:02.132 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T18:10:02.132 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T18:10:02.132 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T18:10:02.132 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T18:10:02.132 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytest-6.2.2-7.el9.noarch 2024-09-10T18:10:02.132 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T18:10:02.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T18:10:02.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T18:10:02.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T18:10:02.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T18:10:02.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T18:10:02.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T18:10:02.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T18:10:02.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T18:10:02.133 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T18:10:02.134 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T18:10:02.134 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T18:10:02.134 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T18:10:02.134 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T18:10:02.134 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T18:10:02.134 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.134 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.134 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:02.134 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T18:10:02.134 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T18:10:02.134 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T18:10:02.134 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:10:02.134 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-10T18:10:02.445 DEBUG:teuthology.parallel:result is None 2024-09-10T18:10:04.352 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 117/117 2024-09-10T18:10:04.353 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/117 2024-09-10T18:10:04.353 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2/117 2024-09-10T18:10:04.353 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 3/117 2024-09-10T18:10:04.353 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 4/117 2024-09-10T18:10:04.353 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57 5/117 2024-09-10T18:10:04.353 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 6/117 2024-09-10T18:10:04.353 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 7/117 2024-09-10T18:10:04.353 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 8/117 2024-09-10T18:10:04.353 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 9/117 2024-09-10T18:10:04.353 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 10/117 2024-09-10T18:10:04.354 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 11/117 2024-09-10T18:10:04.354 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 12/117 2024-09-10T18:10:04.354 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_ 13/117 2024-09-10T18:10:04.354 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 14/117 2024-09-10T18:10:04.354 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 15/117 2024-09-10T18:10:04.354 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_6 16/117 2024-09-10T18:10:04.354 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86 17/117 2024-09-10T18:10:04.354 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 18/117 2024-09-10T18:10:04.354 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el 19/117 2024-09-10T18:10:04.354 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9. 20/117 2024-09-10T18:10:04.354 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_6 21/117 2024-09-10T18:10:04.355 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 22/117 2024-09-10T18:10:04.355 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 23/117 2024-09-10T18:10:04.355 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/117 2024-09-10T18:10:04.355 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 25/117 2024-09-10T18:10:04.355 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 26/117 2024-09-10T18:10:04.355 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 27/117 2024-09-10T18:10:04.355 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395. 28/117 2024-09-10T18:10:04.355 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noa 29/117 2024-09-10T18:10:04.355 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.n 30/117 2024-09-10T18:10:04.364 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be 31/117 2024-09-10T18:10:04.364 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el 32/117 2024-09-10T18:10:04.364 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 33/117 2024-09-10T18:10:04.364 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.e 34/117 2024-09-10T18:10:04.364 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 35/117 2024-09-10T18:10:04.364 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 36/117 2024-09-10T18:10:04.364 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 37/117 2024-09-10T18:10:04.364 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 38/117 2024-09-10T18:10:04.364 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 39/117 2024-09-10T18:10:04.364 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/117 2024-09-10T18:10:04.365 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/117 2024-09-10T18:10:04.365 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/117 2024-09-10T18:10:04.365 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/117 2024-09-10T18:10:04.365 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/117 2024-09-10T18:10:04.365 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 45/117 2024-09-10T18:10:04.365 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 46/117 2024-09-10T18:10:04.365 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 47/117 2024-09-10T18:10:04.365 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 48/117 2024-09-10T18:10:04.365 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/117 2024-09-10T18:10:04.365 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/117 2024-09-10T18:10:04.366 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/117 2024-09-10T18:10:04.366 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/117 2024-09-10T18:10:04.366 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 53/117 2024-09-10T18:10:04.366 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 54/117 2024-09-10T18:10:04.366 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 55/117 2024-09-10T18:10:04.366 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 56/117 2024-09-10T18:10:04.366 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 57/117 2024-09-10T18:10:04.366 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 58/117 2024-09-10T18:10:04.366 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 59/117 2024-09-10T18:10:04.366 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 60/117 2024-09-10T18:10:04.366 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 61/117 2024-09-10T18:10:04.367 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 62/117 2024-09-10T18:10:04.367 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 63/117 2024-09-10T18:10:04.367 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 64/117 2024-09-10T18:10:04.367 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 65/117 2024-09-10T18:10:04.367 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 66/117 2024-09-10T18:10:04.367 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 67/117 2024-09-10T18:10:04.367 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 68/117 2024-09-10T18:10:04.367 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/117 2024-09-10T18:10:04.367 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 70/117 2024-09-10T18:10:04.367 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/117 2024-09-10T18:10:04.368 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/117 2024-09-10T18:10:04.368 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytest-6.2.2-7.el9.noarch 73/117 2024-09-10T18:10:04.368 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 74/117 2024-09-10T18:10:04.368 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 75/117 2024-09-10T18:10:04.368 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 76/117 2024-09-10T18:10:04.368 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 77/117 2024-09-10T18:10:04.368 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 78/117 2024-09-10T18:10:04.368 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 79/117 2024-09-10T18:10:04.368 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 80/117 2024-09-10T18:10:04.368 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 81/117 2024-09-10T18:10:04.369 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 82/117 2024-09-10T18:10:04.369 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 83/117 2024-09-10T18:10:04.369 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 84/117 2024-09-10T18:10:04.369 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 85/117 2024-09-10T18:10:04.369 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 86/117 2024-09-10T18:10:04.369 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 87/117 2024-09-10T18:10:04.369 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 88/117 2024-09-10T18:10:04.369 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 89/117 2024-09-10T18:10:04.369 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 90/117 2024-09-10T18:10:04.369 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 91/117 2024-09-10T18:10:04.369 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 92/117 2024-09-10T18:10:04.370 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 93/117 2024-09-10T18:10:04.370 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 94/117 2024-09-10T18:10:04.370 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 95/117 2024-09-10T18:10:04.370 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 96/117 2024-09-10T18:10:04.370 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 97/117 2024-09-10T18:10:04.370 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 98/117 2024-09-10T18:10:04.370 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 99/117 2024-09-10T18:10:04.370 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 100/117 2024-09-10T18:10:04.370 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 101/117 2024-09-10T18:10:04.370 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 102/117 2024-09-10T18:10:04.371 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 103/117 2024-09-10T18:10:04.371 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 104/117 2024-09-10T18:10:04.371 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 105/117 2024-09-10T18:10:04.371 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 106/117 2024-09-10T18:10:04.371 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 107/117 2024-09-10T18:10:04.371 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 108/117 2024-09-10T18:10:04.371 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 109/117 2024-09-10T18:10:04.371 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 110/117 2024-09-10T18:10:04.371 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 111/117 2024-09-10T18:10:04.371 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 112/117 2024-09-10T18:10:04.371 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 113/117 2024-09-10T18:10:04.371 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 114/117 2024-09-10T18:10:04.372 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 115/117 2024-09-10T18:10:04.372 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 116/117 2024-09-10T18:10:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 117/117 2024-09-10T18:10:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:10:06.593 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2024-09-10T18:10:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.593 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.594 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-09-10T18:10:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:06.594 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:10:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T18:10:06.595 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T18:10:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T18:10:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T18:10:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T18:10:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T18:10:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T18:10:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T18:10:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T18:10:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T18:10:06.596 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T18:10:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T18:10:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T18:10:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T18:10:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T18:10:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T18:10:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T18:10:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T18:10:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T18:10:06.597 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T18:10:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T18:10:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T18:10:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T18:10:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T18:10:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T18:10:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T18:10:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T18:10:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2024-09-10T18:10:06.598 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T18:10:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T18:10:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T18:10:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T18:10:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T18:10:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T18:10:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T18:10:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T18:10:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T18:10:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T18:10:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T18:10:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T18:10:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T18:10:06.599 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T18:10:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T18:10:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T18:10:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T18:10:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T18:10:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T18:10:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T18:10:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T18:10:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pluggy-0.13.1-7.el9.noarch 2024-09-10T18:10:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T18:10:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T18:10:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T18:10:06.600 INFO:teuthology.orchestra.run.smithi133.stdout: python3-py-1.10.0-6.el9.noarch 2024-09-10T18:10:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T18:10:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T18:10:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T18:10:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T18:10:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytest-6.2.2-7.el9.noarch 2024-09-10T18:10:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T18:10:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T18:10:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T18:10:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T18:10:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.601 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:10:06.602 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T18:10:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T18:10:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T18:10:06.603 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:10:06.603 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-09-10T18:10:06.909 DEBUG:teuthology.parallel:result is None 2024-09-10T18:10:06.910 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:10:07.069 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T18:10:07.096 INFO:teuthology.orchestra.run.smithi099.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T18:10:07.096 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T18:10:07.097 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T18:10:07.098 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:10:07.239 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T18:10:07.261 INFO:teuthology.orchestra.run.smithi133.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T18:10:07.261 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T18:10:07.261 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T18:10:07.263 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T18:10:07.263 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:10:07.263 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:10:07.295 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T18:10:07.295 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:10:07.328 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T18:10:07.329 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:10:07.329 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:10:07.364 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:10:07.431 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T18:10:07.431 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:10:07.456 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:10:07.519 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T18:10:07.520 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:10:07.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:10:07.546 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:10:07.612 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T18:10:07.612 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:10:07.637 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:10:07.701 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T18:10:07.701 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:10:07.701 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:10:07.727 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:10:07.791 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T18:10:07.792 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:10:07.816 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:10:07.881 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T18:10:07.976 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)'], 'mon_bind_msgr2': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'cephadm_mode': 'root'} 2024-09-10T18:10:07.976 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:10:07.977 INFO:tasks.cephadm:Cluster fsid is e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:10:07.977 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T18:10:07.977 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.99:6789]', 'mon.c': '[v1:172.21.15.99:6790]', 'mon.b': '[v1:172.21.15.133:6789]'} 2024-09-10T18:10:07.977 INFO:tasks.cephadm:First mon is mon.a on smithi099 2024-09-10T18:10:07.977 INFO:tasks.cephadm:First mgr is y 2024-09-10T18:10:07.977 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T18:10:07.978 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname $(hostname -s) 2024-09-10T18:10:08.003 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname $(hostname -s) 2024-09-10T18:10:08.028 INFO:tasks.cephadm:Downloading cephadm (repo https://git.ceph.com/ceph-ci.git ref 5be57395ea235535910512ae73ba8e7ff25fe588)... 2024-09-10T18:10:08.028 DEBUG:teuthology.orchestra.run.smithi099:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show 5be57395ea235535910512ae73ba8e7ff25fe588:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T18:10:08.059 INFO:teuthology.orchestra.run.smithi099.stderr:Cloning into 'testrepo'... 2024-09-10T18:11:05.790 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 10 18:11 /home/ubuntu/cephtest/cephadm 2024-09-10T18:11:05.791 DEBUG:teuthology.orchestra.run.smithi133:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show 5be57395ea235535910512ae73ba8e7ff25fe588:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T18:11:05.811 INFO:teuthology.orchestra.run.smithi133.stderr:Cloning into 'testrepo'... 2024-09-10T18:12:02.509 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 10 18:12 /home/ubuntu/cephtest/cephadm 2024-09-10T18:12:02.510 DEBUG:teuthology.orchestra.run.smithi099:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T18:12:02.530 DEBUG:teuthology.orchestra.run.smithi133:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T18:12:02.577 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 on all hosts... 2024-09-10T18:12:02.578 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:12:02.580 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:12:04.111 INFO:teuthology.orchestra.run.smithi099.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:12:06.016 INFO:teuthology.orchestra.run.smithi133.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:12:29.914 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T18:12:29.914 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:12:29.914 INFO:teuthology.orchestra.run.smithi099.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:12:29.914 INFO:teuthology.orchestra.run.smithi099.stdout: "repo_digests": [ 2024-09-10T18:12:29.915 INFO:teuthology.orchestra.run.smithi099.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:12:29.915 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T18:12:29.915 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T18:12:34.860 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-10T18:12:34.860 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:12:34.861 INFO:teuthology.orchestra.run.smithi133.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:12:34.861 INFO:teuthology.orchestra.run.smithi133.stdout: "repo_digests": [ 2024-09-10T18:12:34.861 INFO:teuthology.orchestra.run.smithi133.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:12:34.861 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-10T18:12:34.861 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-10T18:12:34.879 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph 2024-09-10T18:12:34.912 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph 2024-09-10T18:12:34.940 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /etc/ceph 2024-09-10T18:12:34.981 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /etc/ceph 2024-09-10T18:12:35.006 INFO:tasks.cephadm:Writing seed config... 2024-09-10T18:12:35.007 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T18:12:35.007 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T18:12:35.007 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-10T18:12:35.007 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-09-10T18:12:35.007 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-10T18:12:35.007 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T18:12:35.007 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T18:12:35.008 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T18:12:35.008 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T18:12:35.008 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T18:12:35.008 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T18:12:35.008 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:12:35.008 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T18:12:35.038 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e9807d90-6f9f-11ef-bcea-c7b262605968 mon election default strategy = 3 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-10T18:12:35.039 DEBUG:teuthology.orchestra.run.smithi099:mon.a> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T18:12:35.081 DEBUG:teuthology.orchestra.run.smithi099:mgr.y> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T18:12:35.124 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T18:12:35.125 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 -v bootstrap --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.99:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:12:35.323 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:12:35.323 INFO:teuthology.orchestra.run.smithi099.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588', '-v', 'bootstrap', '--fsid', 'e9807d90-6f9f-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.99:6789]', '--skip-admin-label'] 2024-09-10T18:12:35.346 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:12:35.346 INFO:teuthology.orchestra.run.smithi099.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T18:12:35.346 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying podman|docker is present... 2024-09-10T18:12:35.368 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:12:35.368 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying lvm2 is present... 2024-09-10T18:12:35.368 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying time synchronization is in place... 2024-09-10T18:12:35.376 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:12:35.376 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:12:35.383 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:12:35.383 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-09-10T18:12:35.390 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout enabled 2024-09-10T18:12:35.396 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout active 2024-09-10T18:12:35.396 INFO:teuthology.orchestra.run.smithi099.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:12:35.397 INFO:teuthology.orchestra.run.smithi099.stdout:Repeating the final host check... 2024-09-10T18:12:35.418 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:12:35.418 INFO:teuthology.orchestra.run.smithi099.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T18:12:35.418 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl is present 2024-09-10T18:12:35.418 INFO:teuthology.orchestra.run.smithi099.stdout:lvcreate is present 2024-09-10T18:12:35.425 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:12:35.425 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:12:35.432 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:12:35.432 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-09-10T18:12:35.439 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout enabled 2024-09-10T18:12:35.445 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout active 2024-09-10T18:12:35.446 INFO:teuthology.orchestra.run.smithi099.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:12:35.446 INFO:teuthology.orchestra.run.smithi099.stdout:Host looks OK 2024-09-10T18:12:35.446 INFO:teuthology.orchestra.run.smithi099.stdout:Cluster fsid: e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:12:35.446 INFO:teuthology.orchestra.run.smithi099.stdout:Acquiring lock 139928788923296 on /run/cephadm/e9807d90-6f9f-11ef-bcea-c7b262605968.lock 2024-09-10T18:12:35.446 INFO:teuthology.orchestra.run.smithi099.stdout:Lock 139928788923296 acquired on /run/cephadm/e9807d90-6f9f-11ef-bcea-c7b262605968.lock 2024-09-10T18:12:35.446 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying IP 172.21.15.99 port 6789 ... 2024-09-10T18:12:35.446 INFO:teuthology.orchestra.run.smithi099.stdout:Base mon IP(s) is [172.21.15.99:6789], mon addrv is [v1:172.21.15.99:6789] 2024-09-10T18:12:35.450 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.99 metric 100 2024-09-10T18:12:35.450 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.99 metric 100 2024-09-10T18:12:35.453 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T18:12:35.453 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T18:12:35.453 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T18:12:35.455 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T18:12:35.456 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T18:12:35.456 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:12:35.456 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T18:12:35.456 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d803/64 scope link noprefixroute 2024-09-10T18:12:35.456 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:12:35.457 INFO:teuthology.orchestra.run.smithi099.stdout:Mon IP `172.21.15.99` is in CIDR network `172.21.0.0/20` 2024-09-10T18:12:35.457 INFO:teuthology.orchestra.run.smithi099.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20'] 2024-09-10T18:12:35.457 INFO:teuthology.orchestra.run.smithi099.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T18:12:35.457 INFO:teuthology.orchestra.run.smithi099.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:12:35.896 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:12:35.896 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:12:35.896 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T18:12:35.896 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying blob sha256:5df13064c03763eb21a89b455ab82ba4ec435f51a3f340cae76514aaf2996d4c 2024-09-10T18:12:35.896 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T18:12:35.896 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying config sha256:4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:12:35.896 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T18:12:36.801 INFO:teuthology.orchestra.run.smithi099.stdout:ceph: stdout ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:12:36.801 INFO:teuthology.orchestra.run.smithi099.stdout:Ceph version: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:12:36.801 INFO:teuthology.orchestra.run.smithi099.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T18:12:37.765 INFO:teuthology.orchestra.run.smithi099.stdout:stat: stdout 167 167 2024-09-10T18:12:37.765 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial keys... 2024-09-10T18:12:38.714 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQAWjOBmBU4dBBAAv3yz4CTl98NNfIpe1N72/g== 2024-09-10T18:12:39.728 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQAXjOBmAysPAhAAMY8YzF5eaVd4L2Z71+RJnQ== 2024-09-10T18:12:40.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQAYjOBmKB90ABAArHBqD4j6/NEecqK3gcCt1A== 2024-09-10T18:12:40.661 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial monmap... 2024-09-10T18:12:41.524 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:12:41.524 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-10T18:12:41.524 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:12:41.524 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:12:41.525 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool for a [v1:172.21.15.99:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:12:41.525 INFO:teuthology.orchestra.run.smithi099.stdout:setting min_mon_release = octopus 2024-09-10T18:12:41.525 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: set fsid to e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:12:41.525 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:12:41.525 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:12:41.525 INFO:teuthology.orchestra.run.smithi099.stdout:Creating mon... 2024-09-10T18:12:42.481 INFO:teuthology.orchestra.run.smithi099.stdout:create mon.a on 2024-09-10T18:12:42.686 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T18:12:42.827 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T18:12:42.978 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e9807d90-6f9f-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-e9807d90-6f9f-11ef-bcea-c7b262605968.target. 2024-09-10T18:12:42.978 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e9807d90-6f9f-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-e9807d90-6f9f-11ef-bcea-c7b262605968.target. 2024-09-10T18:12:43.147 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.a 2024-09-10T18:12:43.147 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T18:12:43.317 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e9807d90-6f9f-11ef-bcea-c7b262605968.target.wants/ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@.service. 2024-09-10T18:12:43.864 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:12:43.864 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-09-10T18:12:43.871 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:12:43.871 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-09-10T18:12:43.871 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-09-10T18:12:43.871 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:12:43.872 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mon to start... 2024-09-10T18:12:43.872 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mon... 2024-09-10T18:12:45.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:44 smithi099 ceph-mon[26669]: from='client.? v1:172.21.15.99:0/3874403132' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T18:12:45.111 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T18:12:45.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout id: e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:12:45.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T18:12:45.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-09-10T18:12:45.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout services: 2024-09-10T18:12:45.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.588594s) 2024-09-10T18:12:45.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T18:12:45.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T18:12:45.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-09-10T18:12:45.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout data: 2024-09-10T18:12:45.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T18:12:45.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T18:12:45.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T18:12:45.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T18:12:45.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-09-10T18:12:45.113 INFO:teuthology.orchestra.run.smithi099.stdout:mon is available 2024-09-10T18:12:45.113 INFO:teuthology.orchestra.run.smithi099.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T18:12:46.147 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:45 smithi099 ceph-mon[26669]: from='client.? v1:172.21.15.99:0/32802227' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:12:46.147 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:45 smithi099 ceph-mon[26669]: from='client.? v1:172.21.15.99:0/32802227' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T18:12:46.150 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-09-10T18:12:46.150 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:12:46.150 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout fsid = e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:12:46.151 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:12:46.151 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.99:6789] 2024-09-10T18:12:46.151 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:12:46.151 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:12:46.151 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:12:46.151 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:12:46.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-09-10T18:12:46.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:12:46.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:12:46.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-09-10T18:12:46.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:12:46.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:12:46.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:12:46.154 INFO:teuthology.orchestra.run.smithi099.stdout:Generating new minimal ceph.conf... 2024-09-10T18:12:47.103 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:46 smithi099 ceph-mon[26669]: from='client.? v1:172.21.15.99:0/603744734' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:12:47.106 INFO:teuthology.orchestra.run.smithi099.stdout:Restarting the monitor... 2024-09-10T18:12:47.357 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:47 smithi099 systemd[1]: Stopping Ceph mon.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:12:47.692 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:47 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a[26665]: 2024-09-10T18:12:47.355+0000 7f11a4e04640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:12:47.692 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:47 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a[26665]: 2024-09-10T18:12:47.355+0000 7f11a4e04640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T18:12:47.692 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:47 smithi099 podman[26910]: 2024-09-10 18:12:47.416993681 +0000 UTC m=+0.151199813 container died 8bfa19aee5a4667527022e42454cd77cdb7629c7dc2ed812460abe70716358f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T18:12:48.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:47 smithi099 podman[26910]: 2024-09-10 18:12:47.797258266 +0000 UTC m=+0.531464386 container cleanup 8bfa19aee5a4667527022e42454cd77cdb7629c7dc2ed812460abe70716358f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T18:12:48.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:47 smithi099 bash[26910]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a 2024-09-10T18:12:48.358 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 podman[26922]: 2024-09-10 18:12:48.031481012 +0000 UTC m=+0.617444194 container remove 8bfa19aee5a4667527022e42454cd77cdb7629c7dc2ed812460abe70716358f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T18:12:48.359 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T18:12:48.359 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 systemd[1]: Stopped Ceph mon.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:12:48.359 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 systemd[1]: Starting Ceph mon.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:12:48.645 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 podman[27018]: 2024-09-10 18:12:48.357423362 +0000 UTC m=+0.103130703 container create 10e130d78a7bf7e9e0ab1d68276f964ff9f9c131219556bea1700efac9674144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:12:48.645 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 podman[27018]: 2024-09-10 18:12:48.270145334 +0000 UTC m=+0.015852690 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:12:48.645 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 podman[27018]: 2024-09-10 18:12:48.568558993 +0000 UTC m=+0.314266335 container init 10e130d78a7bf7e9e0ab1d68276f964ff9f9c131219556bea1700efac9674144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:12:48.645 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 podman[27018]: 2024-09-10 18:12:48.571681375 +0000 UTC m=+0.317388719 container start 10e130d78a7bf7e9e0ab1d68276f964ff9f9c131219556bea1700efac9674144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:12:48.645 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:12:48.645 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 2 2024-09-10T18:12:48.645 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: pidfile_write: ignore empty --pid-file 2024-09-10T18:12:48.645 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: load: jerasure load: lrc 2024-09-10T18:12:48.646 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:12:48.646 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:12:48.646 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:12:48.646 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: DB SUMMARY 2024-09-10T18:12:48.646 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: DB Session ID: W42KHWXBKVOGDVTCB3NF 2024-09-10T18:12:48.646 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:12:48.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:12:48.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-10T18:12:48.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T18:12:48.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 81968 ; 2024-09-10T18:12:48.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:12:48.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:12:48.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:12:48.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:12:48.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.env: 0x55562ef19440 2024-09-10T18:12:48.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.fs: Posix File System 2024-09-10T18:12:48.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.info_log: 0x5556300b8e60 2024-09-10T18:12:48.648 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:12:48.648 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.statistics: (nil) 2024-09-10T18:12:48.648 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.use_fsync: 0 2024-09-10T18:12:48.648 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:12:48.648 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:12:48.648 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:12:48.648 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:12:48.648 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:12:48.648 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:12:48.648 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:12:48.649 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:12:48.649 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:12:48.649 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:12:48.649 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:12:48.649 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.db_log_dir: 2024-09-10T18:12:48.649 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-10T18:12:48.649 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:12:48.649 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:12:48.649 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:12:48.649 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:12:48.649 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:12:48.649 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:12:48.649 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:12:48.650 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:12:48.650 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.write_buffer_manager: 0x555630176780 2024-09-10T18:12:48.650 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:12:48.650 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:12:48.650 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:12:48.650 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:12:48.650 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:12:48.650 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:12:48.651 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:12:48.651 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:12:48.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:12:48.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.unordered_write: 0 2024-09-10T18:12:48.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:12:48.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:12:48.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:12:48.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:12:48.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.row_cache: None 2024-09-10T18:12:48.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.wal_filter: None 2024-09-10T18:12:48.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:12:48.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:12:48.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:12:48.653 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:12:48.653 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:12:48.653 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:12:48.653 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:12:48.653 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:12:48.653 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:12:48.653 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:12:48.653 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:12:48.653 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:12:48.653 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:12:48.653 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:12:48.653 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_open_files: -1 2024-09-10T18:12:48.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:12:48.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:12:48.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:12:48.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:12:48.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:12:48.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Compression algorithms supported: 2024-09-10T18:12:48.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: kZSTD supported: 0 2024-09-10T18:12:48.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:12:48.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:12:48.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:12:48.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:12:48.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:12:48.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:12:48.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:12:48.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:12:48.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-10T18:12:48.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:12:48.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:12:48.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.merge_operator: 2024-09-10T18:12:48.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compaction_filter: None 2024-09-10T18:12:48.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:12:48.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:12:48.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:12:48.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:12:48.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55563005fa38) 2024-09-10T18:12:48.657 INFO:journalctl@ceph.mon.a.smithi099.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:12:48.657 INFO:journalctl@ceph.mon.a.smithi099.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:12:48.657 INFO:journalctl@ceph.mon.a.smithi099.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:12:48.657 INFO:journalctl@ceph.mon.a.smithi099.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:12:48.657 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_type: 0 2024-09-10T18:12:48.658 INFO:journalctl@ceph.mon.a.smithi099.stdout: data_block_index_type: 0 2024-09-10T18:12:48.658 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_shortening: 1 2024-09-10T18:12:48.658 INFO:journalctl@ceph.mon.a.smithi099.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:12:48.658 INFO:journalctl@ceph.mon.a.smithi099.stdout: hash_index_allow_collision: 1 2024-09-10T18:12:48.658 INFO:journalctl@ceph.mon.a.smithi099.stdout: checksum: 1 2024-09-10T18:12:48.658 INFO:journalctl@ceph.mon.a.smithi099.stdout: no_block_cache: 0 2024-09-10T18:12:48.658 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache: 0x5556300951f0 2024-09-10T18:12:48.658 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:12:48.658 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_options: 2024-09-10T18:12:48.658 INFO:journalctl@ceph.mon.a.smithi099.stdout: capacity : 536870912 2024-09-10T18:12:48.658 INFO:journalctl@ceph.mon.a.smithi099.stdout: num_shard_bits : 4 2024-09-10T18:12:48.658 INFO:journalctl@ceph.mon.a.smithi099.stdout: strict_capacity_limit : 0 2024-09-10T18:12:48.658 INFO:journalctl@ceph.mon.a.smithi099.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:12:48.659 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_compressed: (nil) 2024-09-10T18:12:48.659 INFO:journalctl@ceph.mon.a.smithi099.stdout: persistent_cache: (nil) 2024-09-10T18:12:48.659 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_size: 4096 2024-09-10T18:12:48.659 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_size_deviation: 10 2024-09-10T18:12:48.659 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_restart_interval: 16 2024-09-10T18:12:48.659 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_block_restart_interval: 1 2024-09-10T18:12:48.659 INFO:journalctl@ceph.mon.a.smithi099.stdout: metadata_block_size: 4096 2024-09-10T18:12:48.659 INFO:journalctl@ceph.mon.a.smithi099.stdout: partition_filters: 0 2024-09-10T18:12:48.659 INFO:journalctl@ceph.mon.a.smithi099.stdout: use_delta_encoding: 1 2024-09-10T18:12:48.659 INFO:journalctl@ceph.mon.a.smithi099.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:12:48.659 INFO:journalctl@ceph.mon.a.smithi099.stdout: whole_key_filtering: 1 2024-09-10T18:12:48.659 INFO:journalctl@ceph.mon.a.smithi099.stdout: verify_compression: 0 2024-09-10T18:12:48.659 INFO:journalctl@ceph.mon.a.smithi099.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:12:48.660 INFO:journalctl@ceph.mon.a.smithi099.stdout: format_version: 4 2024-09-10T18:12:48.660 INFO:journalctl@ceph.mon.a.smithi099.stdout: enable_index_compression: 1 2024-09-10T18:12:48.660 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_align: 0 2024-09-10T18:12:48.660 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:12:48.660 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:12:48.660 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compression: NoCompression 2024-09-10T18:12:48.660 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:12:48.660 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:12:48.660 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:12:48.660 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.num_levels: 7 2024-09-10T18:12:48.660 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:12:48.661 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:12:48.661 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:12:48.661 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:12:48.661 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:12:48.661 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:12:48.661 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:12:48.661 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:12:48.661 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:12:48.661 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:12:48.661 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:12:48.662 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:12:48.662 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:12:48.662 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:12:48.662 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:12:48.662 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:12:48.663 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:12:48.663 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:12:48.663 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:12:48.663 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:12:48.663 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:12:48.663 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:12:48.664 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:12:48.664 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:12:48.664 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:12:48.664 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:12:48.664 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:12:48.664 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:12:48.664 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:12:48.664 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:12:48.664 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:12:48.664 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:12:48.664 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:12:48.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:12:48.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:12:48.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:12:48.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:12:48.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:12:48.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:12:48.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:12:48.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:12:48.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:12:48.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:12:48.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:12:48.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:12:48.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:12:48.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:12:48.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:12:48.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:12:48.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:12:48.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:12:48.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:12:48.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:12:48.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:12:48.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:12:48.668 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:12:48.668 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:12:48.668 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:12:48.668 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:12:48.668 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:12:48.669 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:12:48.669 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.ttl: 2592000 2024-09-10T18:12:48.669 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:12:48.669 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:12:48.669 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:12:48.669 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:12:48.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:12:48.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:12:48.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:12:48.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:12:48.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-10T18:12:48.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-10T18:12:48.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725991968594979, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-10T18:12:48.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-10T18:12:48.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:12:48.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725991968595712, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 78567, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 76829, "index_size": 200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9969, "raw_average_key_size": 47, "raw_value_size": 71151, "raw_average_value_size": 337, "num_data_blocks": 10, "num_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725991968, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "f2e29f45-878b-4453-82e9-3546eb375df0", "db_session_id": "W42KHWXBKVOGDVTCB3NF"}} 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725991968596130, "job": 1, "event": "recovery_finished"} 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5556300c8700 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: DB pointer 0x555630098000 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** DB Stats ** 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:12:48.671 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:12:48.672 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:12:48.672 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-09-10T18:12:48.672 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** Compaction Stats [default] ** 2024-09-10T18:12:48.672 INFO:journalctl@ceph.mon.a.smithi099.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:12:48.672 INFO:journalctl@ceph.mon.a.smithi099.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:12:48.672 INFO:journalctl@ceph.mon.a.smithi099.stdout: L0 2/0 78.48 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:12:48.672 INFO:journalctl@ceph.mon.a.smithi099.stdout: Sum 2/0 78.48 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:12:48.672 INFO:journalctl@ceph.mon.a.smithi099.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:12:48.672 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-09-10T18:12:48.673 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** Compaction Stats [default] ** 2024-09-10T18:12:48.673 INFO:journalctl@ceph.mon.a.smithi099.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:12:48.673 INFO:journalctl@ceph.mon.a.smithi099.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:12:48.673 INFO:journalctl@ceph.mon.a.smithi099.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 155.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:12:48.673 INFO:journalctl@ceph.mon.a.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:12:48.673 INFO:journalctl@ceph.mon.a.smithi099.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:12:48.673 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:12:48.673 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:12:48.674 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:12:48.674 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:12:48.674 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative compaction: 0.00 GB write, 23.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:12:48.674 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval compaction: 0.00 GB write, 23.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:12:48.674 INFO:journalctl@ceph.mon.a.smithi099.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:12:48.674 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-09-10T18:12:48.674 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:12:48.674 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-09-10T18:12:48.674 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** Compaction Stats [default] ** 2024-09-10T18:12:48.674 INFO:journalctl@ceph.mon.a.smithi099.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:12:48.674 INFO:journalctl@ceph.mon.a.smithi099.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:12:48.674 INFO:journalctl@ceph.mon.a.smithi099.stdout: L0 2/0 78.48 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:12:48.674 INFO:journalctl@ceph.mon.a.smithi099.stdout: Sum 2/0 78.48 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:12:48.675 INFO:journalctl@ceph.mon.a.smithi099.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:12:48.675 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-09-10T18:12:48.675 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** Compaction Stats [default] ** 2024-09-10T18:12:48.675 INFO:journalctl@ceph.mon.a.smithi099.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:12:48.675 INFO:journalctl@ceph.mon.a.smithi099.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:12:48.675 INFO:journalctl@ceph.mon.a.smithi099.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 155.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:12:48.675 INFO:journalctl@ceph.mon.a.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:12:48.675 INFO:journalctl@ceph.mon.a.smithi099.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:12:48.675 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:12:48.675 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:12:48.675 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:12:48.675 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:12:48.675 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative compaction: 0.00 GB write, 22.84 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:12:48.676 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:12:48.676 INFO:journalctl@ceph.mon.a.smithi099.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:12:48.676 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-09-10T18:12:48.676 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:12:48.676 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: starting mon.a rank 0 at public addrs v1:172.21.15.99:6789/0 at bind addrs v1:172.21.15.99:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:12:48.676 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: mon.a@-1(???) e1 preinit fsid e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:12:48.676 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: mon.a@-1(???).mds e1 new map 2024-09-10T18:12:48.676 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: mon.a@-1(???).mds e1 print_map 2024-09-10T18:12:48.676 INFO:journalctl@ceph.mon.a.smithi099.stdout: e1 2024-09-10T18:12:48.677 INFO:journalctl@ceph.mon.a.smithi099.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:12:48.677 INFO:journalctl@ceph.mon.a.smithi099.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:12:48.677 INFO:journalctl@ceph.mon.a.smithi099.stdout: legacy client fscid: -1 2024-09-10T18:12:48.677 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-09-10T18:12:48.677 INFO:journalctl@ceph.mon.a.smithi099.stdout: No filesystems configured 2024-09-10T18:12:48.677 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:12:48.678 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:12:48.678 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:12:48.678 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:12:48.678 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T18:12:48.678 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 bash[27018]: 10e130d78a7bf7e9e0ab1d68276f964ff9f9c131219556bea1700efac9674144 2024-09-10T18:12:48.678 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 systemd[1]: Started Ceph mon.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:12:48.678 INFO:teuthology.orchestra.run.smithi099.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T18:12:49.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:12:49.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: monmap e1: 1 mons at {a=v1:172.21.15.99:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:12:49.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: fsmap 2024-09-10T18:12:49.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:12:49.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:48 smithi099 ceph-mon[27037]: mgrmap e1: no daemons active 2024-09-10T18:12:50.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:50 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2421066690' entity='client.admin' 2024-09-10T18:12:55.809 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T18:12:55.809 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:12:55.809 INFO:teuthology.orchestra.run.smithi099.stdout:Creating mgr... 2024-09-10T18:12:55.809 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T18:12:55.809 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T18:12:56.135 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mgr.y 2024-09-10T18:12:56.142 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-10T18:12:56.555 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e9807d90-6f9f-11ef-bcea-c7b262605968.target.wants/ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@.service. 2024-09-10T18:12:56.958 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:56 smithi099 podman[27248]: 2024-09-10 18:12:56.68748457 +0000 UTC m=+0.015409804 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:12:56.959 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:56 smithi099 podman[27248]: 2024-09-10 18:12:56.957140503 +0000 UTC m=+0.285065783 container create 8da925e9c2a9f22773cd5ce8f9b7f4a9aa98b8791e1655dbbf1b4dddbc25e4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T18:12:57.236 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:57 smithi099 podman[27248]: 2024-09-10 18:12:57.180799144 +0000 UTC m=+0.508724379 container init 8da925e9c2a9f22773cd5ce8f9b7f4a9aa98b8791e1655dbbf1b4dddbc25e4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T18:12:57.236 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:57 smithi099 podman[27248]: 2024-09-10 18:12:57.18351584 +0000 UTC m=+0.511441079 container start 8da925e9c2a9f22773cd5ce8f9b7f4a9aa98b8791e1655dbbf1b4dddbc25e4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T18:12:57.249 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:12:57.249 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-09-10T18:12:57.257 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:12:57.257 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-09-10T18:12:57.257 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-09-10T18:12:57.257 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:12:57.266 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:12:57.266 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-09-10T18:12:57.274 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:12:57.274 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-09-10T18:12:57.274 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-09-10T18:12:57.275 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T18:12:57.275 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr to start... 2024-09-10T18:12:57.275 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr... 2024-09-10T18:12:57.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:57 smithi099 bash[27248]: 8da925e9c2a9f22773cd5ce8f9b7f4a9aa98b8791e1655dbbf1b4dddbc25e4b4 2024-09-10T18:12:57.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:57 smithi099 systemd[1]: Started Ceph mgr.y for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:12:57.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:57 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:12:57.300+0000 7fb7dd1e7ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:12:57.506 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:57 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:12:57.356+0000 7fb7dd1e7ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:12:57.506 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:57 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:12:57.421+0000 7fb7dd1e7ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:12:57.797 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:57 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:12:57.548+0000 7fb7dd1e7ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:12:57.797 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:57 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:12:57.687+0000 7fb7dd1e7ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:12:58.142 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:12:57 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/50732112' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:12:58.142 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:57 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:12:57.954+0000 7fb7dd1e7ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:12:58.143 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:58 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:12:58.015+0000 7fb7dd1e7ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:12:58.426 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:58 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:12:58.178+0000 7fb7dd1e7ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:12:58.428 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-09-10T18:12:58.434 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:12:58.434 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "e9807d90-6f9f-11ef-bcea-c7b262605968", 2024-09-10T18:12:58.434 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:12:58.434 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:12:58.434 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:12:58.435 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:12:58.435 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:12:58.435 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:12:58.435 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:12:58.435 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:12:58.435 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:12:58.435 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:12:58.435 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:12:58.435 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:12:58.435 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-10T18:12:58.435 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:12:58.435 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:12:58.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:12:58.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:12:58.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:12:58.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:12:58.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:12:58.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:12:58.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:12:58.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:12:58.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:12:58.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:12:58.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:12:58.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:12:58.436 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:12:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:12:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:12:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:12:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:12:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:12:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:12:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:12:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:12:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:12:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:12:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:12:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:12:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:12:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:12:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:12:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:12:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:12:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:12:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:12:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:12:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:12:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:12:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:12:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:12:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:12:58.438 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:12:58.439 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:12:43.787558+0000", 2024-09-10T18:12:58.439 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:12:58.439 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:12:58.439 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:12:58.439 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:12:58.439 INFO:teuthology.orchestra.run.smithi099.stdout:mgr not available, waiting (1/15)... 2024-09-10T18:12:59.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:59 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:12:59.017+0000 7fb7dd1e7ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:12:59.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:59 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:12:59.081+0000 7fb7dd1e7ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:13:00.073 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:59 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:12:59.784+0000 7fb7dd1e7ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:13:00.073 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:12:59 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:12:59.894+0000 7fb7dd1e7ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:13:00.073 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:00 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:00.013+0000 7fb7dd1e7ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:13:00.452 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:00 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:00.071+0000 7fb7dd1e7ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:13:00.452 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:00 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:00.195+0000 7fb7dd1e7ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:13:00.453 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:00 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:00.260+0000 7fb7dd1e7ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:13:00.453 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:00 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:00.317+0000 7fb7dd1e7ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:13:00.453 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:00 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:00.450+0000 7fb7dd1e7ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:13:00.711 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:00 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:00.522+0000 7fb7dd1e7ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:13:00.711 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:00 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:00.588+0000 7fb7dd1e7ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:13:00.711 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:00 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:00.669+0000 7fb7dd1e7ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:13:00.975 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:00 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:00.729+0000 7fb7dd1e7ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:13:00.975 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:00 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:00.786+0000 7fb7dd1e7ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:13:01.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:01 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2792342451' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:13:01.625 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:01 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:01.551+0000 7fb7dd1e7ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:13:01.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-09-10T18:13:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:13:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "e9807d90-6f9f-11ef-bcea-c7b262605968", 2024-09-10T18:13:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:13:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:13:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:13:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:13:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:13:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:13:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:13:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:13:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:13:01.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:13:01.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:13:01.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-09-10T18:13:01.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:13:01.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:13:01.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:13:01.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:13:01.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:01.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:13:01.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:13:01.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:13:01.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:13:01.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:13:01.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:13:01.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:13:01.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:13:01.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:01.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:13:01.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:13:01.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:13:01.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:13:01.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:13:01.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:13:01.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:13:01.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:13:01.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:13:01.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:13:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:13:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:13:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:13:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:13:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:13:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:13:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:13:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:13:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:13:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:13:01.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:13:01.638 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:13:01.638 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:01.638 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:13:01.638 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:13:01.638 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:12:43.787558+0000", 2024-09-10T18:13:01.638 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:13:01.638 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:01.638 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:13:01.638 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:13:01.638 INFO:teuthology.orchestra.run.smithi099.stdout:mgr not available, waiting (2/15)... 2024-09-10T18:13:02.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: Activating manager daemon y 2024-09-10T18:13:02.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: mgrmap e2: y(active, starting, since 0.00264619s) 2024-09-10T18:13:02.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: from='mgr.14100 v1:172.21.15.99:0/2511677631' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:13:02.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: from='mgr.14100 v1:172.21.15.99:0/2511677631' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:13:02.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: from='mgr.14100 v1:172.21.15.99:0/2511677631' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:13:02.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: from='mgr.14100 v1:172.21.15.99:0/2511677631' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:13:02.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: from='mgr.14100 v1:172.21.15.99:0/2511677631' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:13:02.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: from='mgr.14100 v1:172.21.15.99:0/2511677631' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:13:02.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: from='mgr.14100 v1:172.21.15.99:0/2511677631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:13:02.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: from='mgr.14100 v1:172.21.15.99:0/2511677631' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:13:02.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: Manager daemon y is now available 2024-09-10T18:13:02.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: from='mgr.14100 v1:172.21.15.99:0/2511677631' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:13:02.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: from='mgr.14100 v1:172.21.15.99:0/2511677631' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:13:02.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: from='mgr.14100 v1:172.21.15.99:0/2511677631' entity='mgr.y' 2024-09-10T18:13:02.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: from='mgr.14100 v1:172.21.15.99:0/2511677631' entity='mgr.y' 2024-09-10T18:13:02.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:02 smithi099 ceph-mon[27037]: from='mgr.14100 v1:172.21.15.99:0/2511677631' entity='mgr.y' 2024-09-10T18:13:03.883 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:03 smithi099 ceph-mon[27037]: mgrmap e3: y(active, since 1.00651s) 2024-09-10T18:13:04.854 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:04 smithi099 ceph-mon[27037]: mgrmap e4: y(active, since 2s) 2024-09-10T18:13:04.854 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:04 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3429441182' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:13:04.863 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-09-10T18:13:04.863 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:13:04.863 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "e9807d90-6f9f-11ef-bcea-c7b262605968", 2024-09-10T18:13:04.863 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:13:04.863 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:13:04.863 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:13:04.864 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:13:04.864 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:04.864 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:13:04.864 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:13:04.864 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:13:04.864 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:13:04.864 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:13:04.864 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:13:04.864 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:13:04.864 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-09-10T18:13:04.864 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:13:04.865 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:13:04.865 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:13:04.865 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:13:04.865 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:04.865 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:13:04.865 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:13:04.865 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:13:04.865 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:13:04.869 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:13:04.870 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:13:04.870 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:13:04.870 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:13:04.870 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:04.870 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:13:04.870 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:13:04.870 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:13:04.870 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:13:04.870 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:13:04.870 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:13:04.870 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:13:04.870 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:13:04.871 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:13:04.871 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:04.871 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:13:04.871 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:13:04.871 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:13:04.871 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:13:04.871 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:04.871 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:13:04.871 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:13:04.871 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:13:04.871 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:13:04.871 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:13:04.871 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:13:04.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:13:04.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:13:04.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:13:04.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:04.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:13:04.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:13:04.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:12:43.787558+0000", 2024-09-10T18:13:04.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:13:04.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:13:04.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:13:04.872 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:13:04.872 INFO:teuthology.orchestra.run.smithi099.stdout:mgr is available 2024-09-10T18:13:06.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:05 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/281179140' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:13:06.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-09-10T18:13:06.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:13:06.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout fsid = e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:13:06.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:13:06.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:13:06.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:13:06.032 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:13:06.032 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:13:06.032 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-09-10T18:13:06.032 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:13:06.032 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:13:06.032 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-09-10T18:13:06.032 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:13:06.032 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:13:06.032 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:13:06.032 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling cephadm module... 2024-09-10T18:13:06.967 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:06 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2628257430' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T18:13:06.967 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:06 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ignoring --setuser ceph since I am not root 2024-09-10T18:13:06.968 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:06 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ignoring --setgroup ceph since I am not root 2024-09-10T18:13:06.968 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:06 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:06.842+0000 7f16a4b9eec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:13:06.968 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:06 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:06.897+0000 7f16a4b9eec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:13:06.968 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:06 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:06.964+0000 7f16a4b9eec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:13:07.228 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:07 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:07.088+0000 7f16a4b9eec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:13:07.229 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:07 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:07.225+0000 7f16a4b9eec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:13:07.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:07 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2628257430' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T18:13:07.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:07 smithi099 ceph-mon[27037]: mgrmap e5: y(active, since 5s) 2024-09-10T18:13:07.755 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:07 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:07.484+0000 7f16a4b9eec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:13:07.755 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:07 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:07.545+0000 7f16a4b9eec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:13:07.755 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:07 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:07.706+0000 7f16a4b9eec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:13:08.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:08 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3244271816' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:13:08.755 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:08 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:08.566+0000 7f16a4b9eec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:13:08.755 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:08 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:08.630+0000 7f16a4b9eec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:13:09.053 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:13:09.054 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T18:13:09.054 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:13:09.054 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:13:09.054 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:13:09.054 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:13:09.054 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for the mgr to restart... 2024-09-10T18:13:09.054 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr epoch 5... 2024-09-10T18:13:09.350 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:09 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:09.347+0000 7f16a4b9eec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:13:09.643 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:09 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:09.461+0000 7f16a4b9eec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:13:09.643 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:09 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:09.582+0000 7f16a4b9eec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:13:09.643 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:09 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:09.640+0000 7f16a4b9eec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:13:10.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:09 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:09.766+0000 7f16a4b9eec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:13:10.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:09 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:09.831+0000 7f16a4b9eec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:13:10.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:09 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:09.887+0000 7f16a4b9eec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:13:10.284 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:10 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:10.018+0000 7f16a4b9eec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:13:10.284 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:10 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:10.080+0000 7f16a4b9eec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:13:10.284 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:10 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:10.144+0000 7f16a4b9eec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:13:10.284 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:10 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:10.225+0000 7f16a4b9eec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:13:10.284 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:10 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:10.281+0000 7f16a4b9eec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:13:10.755 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:10 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:10.337+0000 7f16a4b9eec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:13:11.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:11.068+0000 7f16a4b9eec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:13:11.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: Active manager daemon y restarted 2024-09-10T18:13:11.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: Activating manager daemon y 2024-09-10T18:13:11.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T18:13:11.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: mgrmap e6: y(active, starting, since 0.00306843s) 2024-09-10T18:13:11.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:13:11.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:13:11.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:13:11.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:13:11.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:13:11.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: Manager daemon y is now available 2024-09-10T18:13:11.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' 2024-09-10T18:13:11.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' 2024-09-10T18:13:11.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:13:11.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:13:11.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:13:11.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:11 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:13:12.380 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-mon[27037]: Found migration_current of "None". Setting to last migration. 2024-09-10T18:13:12.381 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-mon[27037]: mgrmap e7: y(active, since 1.00526s) 2024-09-10T18:13:12.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:13:12.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T18:13:12.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:13:12.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:13:12.693 INFO:teuthology.orchestra.run.smithi099.stdout:mgr epoch 5 is available 2024-09-10T18:13:12.693 INFO:teuthology.orchestra.run.smithi099.stdout:Setting orchestrator backend to cephadm... 2024-09-10T18:13:13.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:12.875+0000 7f1656f8c640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:13:12] ENGINE Error in HTTPServer.serve 2024-09-10T18:13:13.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: Traceback (most recent call last): 2024-09-10T18:13:13.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:13:13.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: self._connections.run(self.expiration_interval) 2024-09-10T18:13:13.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:13:13.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: self._run(expiration_interval) 2024-09-10T18:13:13.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:13:13.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:13:13.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:13:13.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:13:13.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:13:13.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: s = self.context.wrap_socket( 2024-09-10T18:13:13.007 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:13:13.007 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: return self.sslsocket_class._create( 2024-09-10T18:13:13.007 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:13:13.007 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: self.do_handshake() 2024-09-10T18:13:13.007 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:13:13.007 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: self._sslobj.do_handshake() 2024-09-10T18:13:13.007 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:13:13.007 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:13.310 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:13 smithi099 ceph-mon[27037]: from='client.14122 v1:172.21.15.99:0/3033795872' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T18:13:13.310 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:13 smithi099 ceph-mon[27037]: from='client.14122 v1:172.21.15.99:0/3033795872' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T18:13:13.310 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:13 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' 2024-09-10T18:13:13.310 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:13 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:13:14.410 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:14 smithi099 ceph-mon[27037]: [10/Sep/2024:18:13:12] ENGINE Bus STARTING 2024-09-10T18:13:14.411 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:14 smithi099 ceph-mon[27037]: [10/Sep/2024:18:13:12] ENGINE Serving on https://172.21.15.99:7150 2024-09-10T18:13:14.411 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:14 smithi099 ceph-mon[27037]: [10/Sep/2024:18:13:12] ENGINE Bus STARTED 2024-09-10T18:13:14.411 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:14 smithi099 ceph-mon[27037]: [10/Sep/2024:18:13:12] ENGINE Error in HTTPServer.serve 2024-09-10T18:13:14.411 INFO:journalctl@ceph.mon.a.smithi099.stdout: Traceback (most recent call last): 2024-09-10T18:13:14.411 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:13:14.411 INFO:journalctl@ceph.mon.a.smithi099.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:13:14.411 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:13:14.411 INFO:journalctl@ceph.mon.a.smithi099.stdout: self._run(expiration_interval) 2024-09-10T18:13:14.411 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:13:14.411 INFO:journalctl@ceph.mon.a.smithi099.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:13:14.411 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:13:14.412 INFO:journalctl@ceph.mon.a.smithi099.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:13:14.412 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:13:14.412 INFO:journalctl@ceph.mon.a.smithi099.stdout: s = self.context.wrap_socket( 2024-09-10T18:13:14.412 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:13:14.412 INFO:journalctl@ceph.mon.a.smithi099.stdout: return self.sslsocket_class._create( 2024-09-10T18:13:14.412 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:13:14.412 INFO:journalctl@ceph.mon.a.smithi099.stdout: self.do_handshake() 2024-09-10T18:13:14.412 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:13:14.413 INFO:journalctl@ceph.mon.a.smithi099.stdout: self._sslobj.do_handshake() 2024-09-10T18:13:14.413 INFO:journalctl@ceph.mon.a.smithi099.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:13:14.413 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:14 smithi099 ceph-mon[27037]: from='client.14130 v1:172.21.15.99:0/2299095625' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:13:14.413 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:14 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' 2024-09-10T18:13:14.413 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:14 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:13:14.413 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:14 smithi099 ceph-mon[27037]: mgrmap e8: y(active, since 2s) 2024-09-10T18:13:15.031 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T18:13:15.031 INFO:teuthology.orchestra.run.smithi099.stdout:Generating ssh key... 2024-09-10T18:13:15.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-mon[27037]: from='client.14132 v1:172.21.15.99:0/2827209870' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:13:16.201 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: Generating public/private rsa key pair. 2024-09-10T18:13:16.202 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: Your identification has been saved in /tmp/tmpjm_t7ya7/key 2024-09-10T18:13:16.202 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: Your public key has been saved in /tmp/tmpjm_t7ya7/key.pub 2024-09-10T18:13:16.202 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: The key fingerprint is: 2024-09-10T18:13:16.202 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: SHA256:M0WDxfX1XNvqiZnpnf78dY7aeV07P8VeZ5RwF0yRYes ceph-e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:13:16.202 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: The key's randomart image is: 2024-09-10T18:13:16.203 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: +---[RSA 3072]----+ 2024-09-10T18:13:16.203 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: | ++.. o*B| 2024-09-10T18:13:16.203 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: | .... o.*B| 2024-09-10T18:13:16.203 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: | . +o*| 2024-09-10T18:13:16.203 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: | . .o.| 2024-09-10T18:13:16.203 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: | S .E | 2024-09-10T18:13:16.204 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: | o * .B| 2024-09-10T18:13:16.204 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: | = ooX| 2024-09-10T18:13:16.204 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: | . o X*| 2024-09-10T18:13:16.204 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: | ooBo@| 2024-09-10T18:13:16.204 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:15 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: +----[SHA256]-----+ 2024-09-10T18:13:16.486 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:16 smithi099 ceph-mon[27037]: from='client.14134 v1:172.21.15.99:0/1381885004' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:13:16.486 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:16 smithi099 ceph-mon[27037]: Generating ssh key... 2024-09-10T18:13:16.486 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:16 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' 2024-09-10T18:13:16.486 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:16 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' 2024-09-10T18:13:17.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdBfTk0Vej22X/O2KNNCjOje2Pw6rvuOSOz0mAPlfB1iE3pqbi9FWsEnURMSfnLNqJpZLtBRMcX8SMDD44XI+mXKBeG9fOkgw3mXB9Hp3BgOanJm6OKIih/p2XoaUxDg6Ia1nK9nTxwBnFRTD1ZxTwQxUMJFkoVpm5669VhuRFrh+Ah+S5q4/24EVkeZNCeMA3Z1dbHr56APd4+zRWGpFB0EiBhFMusk4xE1xVq10rYAn9uv47pfmh40K9kA/bDu0+dWfJCYj/dHswZCsvL15h6JtS2muDFQSZOmY1QzDmH/J/Bhl6yQtrcEWxP6SCO9vJnDB2Tb+cvORIzDzGwVBtYZKCgg6T18v7adMN+Vih64qKKou6bNxcDU3/CaYA0igyAWrOnX8CmeFu6hJH6KVdVBenydqqhQEfdkmJhBAvgyf+S3kxoY1IiO2lk86kbUbJKXmMSannd2XEhxOwh0crKSQ1+0l9cVFwqW5e7s9Lnwplaa0tOnNeGVyBW2iKQE8= ceph-e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:13:17.662 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:13:17.662 INFO:teuthology.orchestra.run.smithi099.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T18:13:17.671 INFO:teuthology.orchestra.run.smithi099.stdout:Adding host smithi099... 2024-09-10T18:13:17.891 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:17 smithi099 ceph-mon[27037]: from='client.14136 v1:172.21.15.99:0/751534664' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:13:19.034 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:18 smithi099 ceph-mon[27037]: from='client.14138 v1:172.21.15.99:0/2505892045' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "addr": "172.21.15.99", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:13:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:19 smithi099 ceph-mon[27037]: Deploying cephadm binary to smithi099 2024-09-10T18:13:21.562 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Added host 'smithi099' with addr '172.21.15.99' 2024-09-10T18:13:21.562 INFO:teuthology.orchestra.run.smithi099.stdout:Deploying unmanaged mon service... 2024-09-10T18:13:22.173 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:21 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' 2024-09-10T18:13:22.173 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:21 smithi099 ceph-mon[27037]: Added host smithi099 2024-09-10T18:13:22.174 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:21 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:13:23.483 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:23 smithi099 ceph-mon[27037]: from='client.14140 v1:172.21.15.99:0/1430956809' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:13:23.483 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:23 smithi099 ceph-mon[27037]: Saving service mon spec with placement count:5 2024-09-10T18:13:23.484 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:23 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' 2024-09-10T18:13:23.484 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:23 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' 2024-09-10T18:13:23.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T18:13:23.812 INFO:teuthology.orchestra.run.smithi099.stdout:Deploying unmanaged mgr service... 2024-09-10T18:13:25.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:24 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' 2024-09-10T18:13:25.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:24 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' 2024-09-10T18:13:25.649 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T18:13:25.898 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:25 smithi099 ceph-mon[27037]: from='client.14142 v1:172.21.15.99:0/3401005997' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:13:25.898 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:25 smithi099 ceph-mon[27037]: Saving service mgr spec with placement count:2 2024-09-10T18:13:25.898 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:25 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' 2024-09-10T18:13:27.198 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:27 smithi099 ceph-mon[27037]: from='mgr.14118 v1:172.21.15.99:0/3888733337' entity='mgr.y' 2024-09-10T18:13:27.198 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:27 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3595766263' entity='client.admin' 2024-09-10T18:13:28.291 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling the dashboard module... 2024-09-10T18:13:28.526 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:28 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3836332956' entity='client.admin' 2024-09-10T18:13:29.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:29 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2720839100' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T18:13:29.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:29 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ignoring --setuser ceph since I am not root 2024-09-10T18:13:29.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:29 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ignoring --setgroup ceph since I am not root 2024-09-10T18:13:29.905 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:29 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:29.522+0000 7fa8714daec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:13:29.906 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:29 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:29.577+0000 7fa8714daec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:13:29.906 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:29 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:29.641+0000 7fa8714daec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:13:29.906 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:29 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:29.768+0000 7fa8714daec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:13:29.906 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:29 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:29.903+0000 7fa8714daec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:13:30.469 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:30 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2720839100' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T18:13:30.469 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:30 smithi099 ceph-mon[27037]: mgrmap e9: y(active, since 18s) 2024-09-10T18:13:30.470 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:30 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:30.166+0000 7fa8714daec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:13:30.470 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:30 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:30.228+0000 7fa8714daec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:13:30.470 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:30 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:30.395+0000 7fa8714daec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:13:31.336 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:13:31.336 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T18:13:31.336 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:13:31.336 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:13:31.336 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:13:31.336 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:13:31.336 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for the mgr to restart... 2024-09-10T18:13:31.336 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr epoch 9... 2024-09-10T18:13:31.574 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:31 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3061898719' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:13:31.575 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:31 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:31.254+0000 7fa8714daec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:13:31.575 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:31 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:31.318+0000 7fa8714daec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:13:32.329 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:32 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:32.034+0000 7fa8714daec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:13:32.329 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:32 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:32.145+0000 7fa8714daec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:13:32.329 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:32 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:32.264+0000 7fa8714daec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:13:32.330 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:32 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:32.322+0000 7fa8714daec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:13:32.707 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:32 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:32.448+0000 7fa8714daec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:13:32.707 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:32 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:32.514+0000 7fa8714daec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:13:32.707 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:32 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:32.571+0000 7fa8714daec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:13:32.707 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:32 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:32.705+0000 7fa8714daec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:13:32.970 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:32 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:32.768+0000 7fa8714daec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:13:32.970 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:32 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:32.832+0000 7fa8714daec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:13:32.971 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:32 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:32.912+0000 7fa8714daec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:13:32.971 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:32 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:32.968+0000 7fa8714daec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:13:33.255 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:33.025+0000 7fa8714daec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:13:34.255 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:33.757+0000 7fa8714daec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:13:34.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-mon[27037]: Active manager daemon y restarted 2024-09-10T18:13:34.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-mon[27037]: Activating manager daemon y 2024-09-10T18:13:34.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-mon[27037]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T18:13:34.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-mon[27037]: mgrmap e10: y(active, starting, since 0.00312305s) 2024-09-10T18:13:34.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:13:34.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:13:34.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:13:34.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:13:34.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:13:34.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-mon[27037]: Manager daemon y is now available 2024-09-10T18:13:34.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:13:34.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:13:34.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:13:35.375 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:35.135+0000 7fa822037640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:13:35] ENGINE Error in HTTPServer.serve 2024-09-10T18:13:35.375 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: Traceback (most recent call last): 2024-09-10T18:13:35.375 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:13:35.376 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: self._connections.run(self.expiration_interval) 2024-09-10T18:13:35.376 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:13:35.376 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: self._run(expiration_interval) 2024-09-10T18:13:35.376 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:13:35.376 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:13:35.376 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:13:35.376 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:13:35.376 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:13:35.376 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: s = self.context.wrap_socket( 2024-09-10T18:13:35.376 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:13:35.376 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: return self.sslsocket_class._create( 2024-09-10T18:13:35.377 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:13:35.377 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: self.do_handshake() 2024-09-10T18:13:35.377 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:13:35.377 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: self._sslobj.do_handshake() 2024-09-10T18:13:35.377 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:13:35.377 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:13:35.378 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:13:35.378 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T18:13:35.378 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:13:35.378 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:13:35.378 INFO:teuthology.orchestra.run.smithi099.stdout:mgr epoch 9 is available 2024-09-10T18:13:35.378 INFO:teuthology.orchestra.run.smithi099.stdout:Generating a dashboard self-signed certificate... 2024-09-10T18:13:35.961 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-mon[27037]: mgrmap e11: y(active, since 1.00579s) 2024-09-10T18:13:35.962 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-mon[27037]: [10/Sep/2024:18:13:35] ENGINE Bus STARTING 2024-09-10T18:13:35.962 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-mon[27037]: [10/Sep/2024:18:13:35] ENGINE Serving on https://172.21.15.99:7150 2024-09-10T18:13:35.962 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-mon[27037]: [10/Sep/2024:18:13:35] ENGINE Bus STARTED 2024-09-10T18:13:35.962 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-mon[27037]: [10/Sep/2024:18:13:35] ENGINE Error in HTTPServer.serve 2024-09-10T18:13:35.962 INFO:journalctl@ceph.mon.a.smithi099.stdout: Traceback (most recent call last): 2024-09-10T18:13:35.962 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:13:35.962 INFO:journalctl@ceph.mon.a.smithi099.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:13:35.963 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:13:35.963 INFO:journalctl@ceph.mon.a.smithi099.stdout: self._run(expiration_interval) 2024-09-10T18:13:35.963 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:13:35.963 INFO:journalctl@ceph.mon.a.smithi099.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:13:35.963 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:13:35.963 INFO:journalctl@ceph.mon.a.smithi099.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:13:35.963 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:13:35.963 INFO:journalctl@ceph.mon.a.smithi099.stdout: s = self.context.wrap_socket( 2024-09-10T18:13:35.964 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:13:35.964 INFO:journalctl@ceph.mon.a.smithi099.stdout: return self.sslsocket_class._create( 2024-09-10T18:13:35.964 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:13:35.964 INFO:journalctl@ceph.mon.a.smithi099.stdout: self.do_handshake() 2024-09-10T18:13:35.964 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:13:35.964 INFO:journalctl@ceph.mon.a.smithi099.stdout: self._sslobj.do_handshake() 2024-09-10T18:13:35.964 INFO:journalctl@ceph.mon.a.smithi099.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:13:35.964 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:35 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:36.868 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T18:13:36.868 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial admin user... 2024-09-10T18:13:37.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:37 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:37.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:37 smithi099 ceph-mon[27037]: from='client.14162 v1:172.21.15.99:0/3059142985' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:13:37.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:37 smithi099 ceph-mon[27037]: mgrmap e12: y(active, since 2s) 2024-09-10T18:13:37.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:37 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:37.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:37 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:37.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:37 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:39.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:38 smithi099 ceph-mon[27037]: from='client.14164 v1:172.21.15.99:0/521248884' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:13:39.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:38 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:39.065 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$QQ/7h0Amw34IUGRoJYqN2OIBncj98Dfajh3aGcyeWBIhfpIPz8cAy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725992017, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T18:13:39.065 INFO:teuthology.orchestra.run.smithi099.stdout:Fetching dashboard port number... 2024-09-10T18:13:39.911 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:39 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/637541369' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T18:13:40.197 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T18:13:40.207 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:13:40.207 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-09-10T18:13:40.214 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:13:40.214 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-09-10T18:13:40.214 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-09-10T18:13:40.214 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T18:13:40.216 INFO:teuthology.orchestra.run.smithi099.stdout:Ceph Dashboard is now available at: 2024-09-10T18:13:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:13:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: URL: https://smithi099.front.sepia.ceph.com:8443/ 2024-09-10T18:13:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: User: admin 2024-09-10T18:13:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: Password: fr1c3exq8d 2024-09-10T18:13:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:13:40.217 INFO:teuthology.orchestra.run.smithi099.stdout:Saving cluster configuration to /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config directory 2024-09-10T18:13:40.217 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling autotune for osd_memory_target 2024-09-10T18:13:41.984 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:41 smithi099 ceph-mon[27037]: mgrmap e13: y(active, since 6s) 2024-09-10T18:13:42.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T18:13:42.661 INFO:teuthology.orchestra.run.smithi099.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T18:13:42.661 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:13:42.662 INFO:teuthology.orchestra.run.smithi099.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:13:42.662 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:13:42.662 INFO:teuthology.orchestra.run.smithi099.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T18:13:42.662 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:13:42.662 INFO:teuthology.orchestra.run.smithi099.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T18:13:42.662 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:13:42.662 INFO:teuthology.orchestra.run.smithi099.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T18:13:42.662 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:13:42.662 INFO:teuthology.orchestra.run.smithi099.stdout: ceph telemetry on 2024-09-10T18:13:42.662 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:13:42.662 INFO:teuthology.orchestra.run.smithi099.stdout:For more information see: 2024-09-10T18:13:42.662 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:13:42.662 INFO:teuthology.orchestra.run.smithi099.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-10T18:13:42.663 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:13:42.663 INFO:teuthology.orchestra.run.smithi099.stdout:Bootstrap complete. 2024-09-10T18:13:42.703 INFO:tasks.cephadm:Fetching config... 2024-09-10T18:13:42.703 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:13:42.703 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T18:13:42.721 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T18:13:42.721 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:13:42.721 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T18:13:42.778 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T18:13:42.778 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:13:42.778 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T18:13:42.845 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T18:13:42.845 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:13:42.845 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T18:13:42.900 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T18:13:42.900 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdBfTk0Vej22X/O2KNNCjOje2Pw6rvuOSOz0mAPlfB1iE3pqbi9FWsEnURMSfnLNqJpZLtBRMcX8SMDD44XI+mXKBeG9fOkgw3mXB9Hp3BgOanJm6OKIih/p2XoaUxDg6Ia1nK9nTxwBnFRTD1ZxTwQxUMJFkoVpm5669VhuRFrh+Ah+S5q4/24EVkeZNCeMA3Z1dbHr56APd4+zRWGpFB0EiBhFMusk4xE1xVq10rYAn9uv47pfmh40K9kA/bDu0+dWfJCYj/dHswZCsvL15h6JtS2muDFQSZOmY1QzDmH/J/Bhl6yQtrcEWxP6SCO9vJnDB2Tb+cvORIzDzGwVBtYZKCgg6T18v7adMN+Vih64qKKou6bNxcDU3/CaYA0igyAWrOnX8CmeFu6hJH6KVdVBenydqqhQEfdkmJhBAvgyf+S3kxoY1IiO2lk86kbUbJKXmMSannd2XEhxOwh0crKSQ1+0l9cVFwqW5e7s9Lnwplaa0tOnNeGVyBW2iKQE8= ceph-e9807d90-6f9f-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:13:42.988 INFO:teuthology.orchestra.run.smithi099.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdBfTk0Vej22X/O2KNNCjOje2Pw6rvuOSOz0mAPlfB1iE3pqbi9FWsEnURMSfnLNqJpZLtBRMcX8SMDD44XI+mXKBeG9fOkgw3mXB9Hp3BgOanJm6OKIih/p2XoaUxDg6Ia1nK9nTxwBnFRTD1ZxTwQxUMJFkoVpm5669VhuRFrh+Ah+S5q4/24EVkeZNCeMA3Z1dbHr56APd4+zRWGpFB0EiBhFMusk4xE1xVq10rYAn9uv47pfmh40K9kA/bDu0+dWfJCYj/dHswZCsvL15h6JtS2muDFQSZOmY1QzDmH/J/Bhl6yQtrcEWxP6SCO9vJnDB2Tb+cvORIzDzGwVBtYZKCgg6T18v7adMN+Vih64qKKou6bNxcDU3/CaYA0igyAWrOnX8CmeFu6hJH6KVdVBenydqqhQEfdkmJhBAvgyf+S3kxoY1IiO2lk86kbUbJKXmMSannd2XEhxOwh0crKSQ1+0l9cVFwqW5e7s9Lnwplaa0tOnNeGVyBW2iKQE8= ceph-e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:13:43.000 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdBfTk0Vej22X/O2KNNCjOje2Pw6rvuOSOz0mAPlfB1iE3pqbi9FWsEnURMSfnLNqJpZLtBRMcX8SMDD44XI+mXKBeG9fOkgw3mXB9Hp3BgOanJm6OKIih/p2XoaUxDg6Ia1nK9nTxwBnFRTD1ZxTwQxUMJFkoVpm5669VhuRFrh+Ah+S5q4/24EVkeZNCeMA3Z1dbHr56APd4+zRWGpFB0EiBhFMusk4xE1xVq10rYAn9uv47pfmh40K9kA/bDu0+dWfJCYj/dHswZCsvL15h6JtS2muDFQSZOmY1QzDmH/J/Bhl6yQtrcEWxP6SCO9vJnDB2Tb+cvORIzDzGwVBtYZKCgg6T18v7adMN+Vih64qKKou6bNxcDU3/CaYA0igyAWrOnX8CmeFu6hJH6KVdVBenydqqhQEfdkmJhBAvgyf+S3kxoY1IiO2lk86kbUbJKXmMSannd2XEhxOwh0crKSQ1+0l9cVFwqW5e7s9Lnwplaa0tOnNeGVyBW2iKQE8= ceph-e9807d90-6f9f-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:13:43.099 INFO:teuthology.orchestra.run.smithi133.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdBfTk0Vej22X/O2KNNCjOje2Pw6rvuOSOz0mAPlfB1iE3pqbi9FWsEnURMSfnLNqJpZLtBRMcX8SMDD44XI+mXKBeG9fOkgw3mXB9Hp3BgOanJm6OKIih/p2XoaUxDg6Ia1nK9nTxwBnFRTD1ZxTwQxUMJFkoVpm5669VhuRFrh+Ah+S5q4/24EVkeZNCeMA3Z1dbHr56APd4+zRWGpFB0EiBhFMusk4xE1xVq10rYAn9uv47pfmh40K9kA/bDu0+dWfJCYj/dHswZCsvL15h6JtS2muDFQSZOmY1QzDmH/J/Bhl6yQtrcEWxP6SCO9vJnDB2Tb+cvORIzDzGwVBtYZKCgg6T18v7adMN+Vih64qKKou6bNxcDU3/CaYA0igyAWrOnX8CmeFu6hJH6KVdVBenydqqhQEfdkmJhBAvgyf+S3kxoY1IiO2lk86kbUbJKXmMSannd2XEhxOwh0crKSQ1+0l9cVFwqW5e7s9Lnwplaa0tOnNeGVyBW2iKQE8= ceph-e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:13:43.113 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T18:13:43.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:42 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/188266849' entity='client.admin' 2024-09-10T18:13:43.357 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:13:46.239 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T18:13:46.239 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T18:13:46.584 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:13:46.861 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:46 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3252406792' entity='client.admin' 2024-09-10T18:13:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:48 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:48 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:48 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:13:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:48 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:48 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:13:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:48 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:49.704 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi133 2024-09-10T18:13:49.704 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T18:13:49.704 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.conf 2024-09-10T18:13:49.723 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T18:13:49.724 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:13:49.782 INFO:tasks.cephadm:Adding host smithi133 to orchestrator... 2024-09-10T18:13:49.782 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch host add smithi133 2024-09-10T18:13:50.025 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:13:50.146 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:49 smithi099 ceph-mon[27037]: from='client.14174 v1:172.21.15.99:0/4156624896' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:13:50.146 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:49 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:50.146 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:49 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:13:50.146 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:49 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:13:50.146 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:49 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:13:50.146 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:49 smithi099 ceph-mon[27037]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-10T18:13:50.146 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:49 smithi099 ceph-mon[27037]: Updating smithi099:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:13:50.146 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:49 smithi099 ceph-mon[27037]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:13:50.146 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:49 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:50.147 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:49 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:50.147 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:49 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:51.076 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:50 smithi099 ceph-mon[27037]: Updating smithi099:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:13:52.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:52 smithi099 ceph-mon[27037]: from='client.14176 v1:172.21.15.99:0/3499087584' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:13:53.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:53 smithi099 ceph-mon[27037]: Deploying cephadm binary to smithi133 2024-09-10T18:13:54.489 INFO:teuthology.orchestra.run.smithi099.stdout:Added host 'smithi133' with addr '172.21.15.133' 2024-09-10T18:13:55.091 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:54 smithi099 ceph-mon[27037]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:13:55.091 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:54 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:55.091 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:54 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:13:55.116 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T18:13:55.352 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:13:56.181 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:55 smithi099 ceph-mon[27037]: Added host smithi133 2024-09-10T18:13:56.181 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:55 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:56.181 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:55 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:57.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:56 smithi099 ceph-mon[27037]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:13:57.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:56 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:57.120 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:13:57.120 INFO:teuthology.orchestra.run.smithi099.stdout:[{"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}] 2024-09-10T18:13:57.712 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T18:13:57.712 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T18:13:57.949 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:13:58.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:57 smithi099 ceph-mon[27037]: from='client.14178 v1:172.21.15.99:0/2280529037' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:13:59.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:58 smithi099 ceph-mon[27037]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:13:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:58 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:58 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:58 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:58 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:13:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:58 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:13:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:58 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:13:59.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:13:58 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:00.440 INFO:teuthology.orchestra.run.smithi099.stderr:adjusted tunables profile to default 2024-09-10T18:14:00.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:00 smithi099 ceph-mon[27037]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T18:14:00.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:00 smithi099 ceph-mon[27037]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:14:00.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:00 smithi099 ceph-mon[27037]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:14:00.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:00 smithi099 ceph-mon[27037]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:14:00.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:00 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:00.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:00 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:00.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:00 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:00.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:00 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/619105202' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:14:01.048 INFO:tasks.cephadm:Adding mon.a on smithi099 2024-09-10T18:14:01.048 INFO:tasks.cephadm:Adding mon.c on smithi099 2024-09-10T18:14:01.048 INFO:tasks.cephadm:Adding mon.b on smithi133 2024-09-10T18:14:01.049 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi099:[v1:172.21.15.99:6789]=a;smithi099:[v1:172.21.15.99:6790]=c;smithi133:[v1:172.21.15.133:6789]=b' 2024-09-10T18:14:01.291 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:14:01.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:01 smithi099 ceph-mon[27037]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:01.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:01 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/619105202' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:14:01.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:01 smithi099 ceph-mon[27037]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:14:02.975 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mon update... 2024-09-10T18:14:03.505 DEBUG:teuthology.orchestra.run.smithi099:mon.c> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T18:14:03.508 DEBUG:teuthology.orchestra.run.smithi133:mon.b> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T18:14:03.510 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:14:03.510 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:14:03.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:03 smithi099 ceph-mon[27037]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:03.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:03 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:03.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:03 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:03.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:03 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:03.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:03 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:03.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:03 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:03.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:03 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:03.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:03 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:14:04.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:04 smithi099 ceph-mon[27037]: from='client.14182 v1:172.21.15.133:0/4097790779' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:[v1:172.21.15.99:6789]=a;smithi099:[v1:172.21.15.99:6790]=c;smithi133:[v1:172.21.15.133:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:14:04.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:04 smithi099 ceph-mon[27037]: Saving service mon spec with placement smithi099:[v1:172.21.15.99:6789]=a;smithi099:[v1:172.21.15.99:6790]=c;smithi133:[v1:172.21.15.133:6789]=b;count:3 2024-09-10T18:14:04.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:04 smithi099 ceph-mon[27037]: Deploying daemon mon.b on smithi133 2024-09-10T18:14:04.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:04 smithi099 ceph-mon[27037]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:07.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 systemd[1]: Starting Ceph mon.b for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:14:07.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:06 smithi099 ceph-mon[27037]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:07.392 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:14:07.392 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":1,"fsid":"e9807d90-6f9f-11ef-bcea-c7b262605968","modified":"2024-09-10T18:12:40.967516Z","created":"2024-09-10T18:12:40.967516Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T18:14:07.392 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 1 2024-09-10T18:14:07.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 podman[31182]: 2024-09-10 18:14:07.314322347 +0000 UTC m=+0.015473451 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:14:07.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 podman[31182]: 2024-09-10 18:14:07.416172074 +0000 UTC m=+0.117323174 container create dc0f0783b0838c574c61cb772c281c9f48525a08266ee53c617decc3dec5c730 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T18:14:08.191 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 podman[31182]: 2024-09-10 18:14:07.787195194 +0000 UTC m=+0.488346294 container init dc0f0783b0838c574c61cb772c281c9f48525a08266ee53c617decc3dec5c730 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-b, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T18:14:08.191 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 podman[31182]: 2024-09-10 18:14:07.790580325 +0000 UTC m=+0.491731433 container start dc0f0783b0838c574c61cb772c281c9f48525a08266ee53c617decc3dec5c730 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:14:08.191 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:14:08.191 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 2 2024-09-10T18:14:08.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: pidfile_write: ignore empty --pid-file 2024-09-10T18:14:08.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: load: jerasure load: lrc 2024-09-10T18:14:08.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:14:08.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:14:08.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:14:08.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: DB SUMMARY 2024-09-10T18:14:08.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: DB Session ID: 4KVRSJGH3A75CSS3Q9KA 2024-09-10T18:14:08.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:14:08.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:14:08.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:14:08.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T18:14:08.192 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 476 ; 2024-09-10T18:14:08.193 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:14:08.193 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:14:08.193 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:14:08.193 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:14:08.193 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.env: 0x555e46ab1440 2024-09-10T18:14:08.193 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.fs: Posix File System 2024-09-10T18:14:08.193 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.info_log: 0x555e48148e20 2024-09-10T18:14:08.193 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:14:08.193 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.statistics: (nil) 2024-09-10T18:14:08.195 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.use_fsync: 0 2024-09-10T18:14:08.195 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:14:08.195 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:14:08.195 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:14:08.195 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:14:08.195 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:14:08.195 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:14:08.196 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:14:08.196 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:14:08.196 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:14:08.196 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:14:08.196 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:14:08.196 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.db_log_dir: 2024-09-10T18:14:08.196 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-09-10T18:14:08.196 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:14:08.196 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:14:08.196 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:14:08.196 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:14:08.196 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:14:08.197 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:14:08.197 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:14:08.197 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:14:08.197 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.write_buffer_manager: 0x555e482067e0 2024-09-10T18:14:08.197 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:14:08.197 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:14:08.197 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:14:08.197 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:14:08.197 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:14:08.197 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:14:08.197 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:14:08.197 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:14:08.198 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:14:08.198 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.unordered_write: 0 2024-09-10T18:14:08.198 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:14:08.198 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:14:08.198 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:14:08.198 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:14:08.198 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.row_cache: None 2024-09-10T18:14:08.198 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.wal_filter: None 2024-09-10T18:14:08.198 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:14:08.198 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:14:08.198 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:14:08.198 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:14:08.199 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:14:08.199 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:14:08.199 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:14:08.199 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:14:08.199 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:14:08.199 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:14:08.199 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:14:08.199 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:14:08.199 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:14:08.199 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:14:08.199 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:14:08.199 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:14:08.199 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:14:08.200 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:14:08.200 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:14:08.200 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:14:08.200 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:14:08.200 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:14:08.200 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:14:08.200 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:14:08.200 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:14:08.200 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:14:08.200 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:14:08.200 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_open_files: -1 2024-09-10T18:14:08.200 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:14:08.200 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:14:08.201 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:14:08.201 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:14:08.201 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:14:08.201 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Compression algorithms supported: 2024-09-10T18:14:08.201 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: kZSTD supported: 0 2024-09-10T18:14:08.201 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:14:08.201 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:14:08.201 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:14:08.201 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:14:08.201 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:14:08.201 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:14:08.201 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:14:08.202 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:14:08.202 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2024-09-10T18:14:08.202 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:14:08.202 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:14:08.202 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.merge_operator: 2024-09-10T18:14:08.202 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compaction_filter: None 2024-09-10T18:14:08.202 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:14:08.202 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:14:08.202 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:14:08.202 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:14:08.202 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555e480efa38) 2024-09-10T18:14:08.202 INFO:journalctl@ceph.mon.b.smithi133.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:14:08.202 INFO:journalctl@ceph.mon.b.smithi133.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:14:08.203 INFO:journalctl@ceph.mon.b.smithi133.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:14:08.203 INFO:journalctl@ceph.mon.b.smithi133.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:14:08.203 INFO:journalctl@ceph.mon.b.smithi133.stdout: index_type: 0 2024-09-10T18:14:08.203 INFO:journalctl@ceph.mon.b.smithi133.stdout: data_block_index_type: 0 2024-09-10T18:14:08.203 INFO:journalctl@ceph.mon.b.smithi133.stdout: index_shortening: 1 2024-09-10T18:14:08.203 INFO:journalctl@ceph.mon.b.smithi133.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:14:08.203 INFO:journalctl@ceph.mon.b.smithi133.stdout: hash_index_allow_collision: 1 2024-09-10T18:14:08.203 INFO:journalctl@ceph.mon.b.smithi133.stdout: checksum: 1 2024-09-10T18:14:08.203 INFO:journalctl@ceph.mon.b.smithi133.stdout: no_block_cache: 0 2024-09-10T18:14:08.203 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_cache: 0x555e481251f0 2024-09-10T18:14:08.203 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:14:08.203 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_cache_options: 2024-09-10T18:14:08.204 INFO:journalctl@ceph.mon.b.smithi133.stdout: capacity : 536870912 2024-09-10T18:14:08.204 INFO:journalctl@ceph.mon.b.smithi133.stdout: num_shard_bits : 4 2024-09-10T18:14:08.204 INFO:journalctl@ceph.mon.b.smithi133.stdout: strict_capacity_limit : 0 2024-09-10T18:14:08.204 INFO:journalctl@ceph.mon.b.smithi133.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:14:08.204 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_cache_compressed: (nil) 2024-09-10T18:14:08.204 INFO:journalctl@ceph.mon.b.smithi133.stdout: persistent_cache: (nil) 2024-09-10T18:14:08.204 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_size: 4096 2024-09-10T18:14:08.204 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_size_deviation: 10 2024-09-10T18:14:08.204 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_restart_interval: 16 2024-09-10T18:14:08.204 INFO:journalctl@ceph.mon.b.smithi133.stdout: index_block_restart_interval: 1 2024-09-10T18:14:08.204 INFO:journalctl@ceph.mon.b.smithi133.stdout: metadata_block_size: 4096 2024-09-10T18:14:08.204 INFO:journalctl@ceph.mon.b.smithi133.stdout: partition_filters: 0 2024-09-10T18:14:08.204 INFO:journalctl@ceph.mon.b.smithi133.stdout: use_delta_encoding: 1 2024-09-10T18:14:08.205 INFO:journalctl@ceph.mon.b.smithi133.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:14:08.205 INFO:journalctl@ceph.mon.b.smithi133.stdout: whole_key_filtering: 1 2024-09-10T18:14:08.205 INFO:journalctl@ceph.mon.b.smithi133.stdout: verify_compression: 0 2024-09-10T18:14:08.205 INFO:journalctl@ceph.mon.b.smithi133.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:14:08.205 INFO:journalctl@ceph.mon.b.smithi133.stdout: format_version: 4 2024-09-10T18:14:08.205 INFO:journalctl@ceph.mon.b.smithi133.stdout: enable_index_compression: 1 2024-09-10T18:14:08.205 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_align: 0 2024-09-10T18:14:08.205 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:14:08.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:14:08.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compression: NoCompression 2024-09-10T18:14:08.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:14:08.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:14:08.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:14:08.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.num_levels: 7 2024-09-10T18:14:08.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:14:08.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:14:08.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:14:08.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:14:08.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:14:08.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:14:08.207 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:14:08.207 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:14:08.207 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:14:08.207 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:14:08.207 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:14:08.207 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:14:08.207 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:14:08.207 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:14:08.207 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:14:08.207 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:14:08.207 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:14:08.207 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:14:08.208 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:14:08.208 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:14:08.208 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:14:08.208 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:14:08.208 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:14:08.208 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:14:08.208 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:14:08.208 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:14:08.208 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:14:08.208 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:14:08.208 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:14:08.208 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:14:08.209 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:14:08.209 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:14:08.209 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:14:08.209 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:14:08.209 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:14:08.209 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:14:08.209 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:14:08.209 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:14:08.209 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:14:08.209 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:14:08.209 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:14:08.209 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:14:08.210 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:14:08.210 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:14:08.210 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:14:08.210 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:14:08.210 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:14:08.210 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:14:08.210 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:14:08.210 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:14:08.210 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:14:08.210 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:14:08.210 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:14:08.210 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:14:08.211 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:14:08.211 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:14:08.211 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:14:08.211 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:14:08.211 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:14:08.211 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:14:08.211 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:14:08.211 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.ttl: 2592000 2024-09-10T18:14:08.211 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:14:08.211 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:14:08.211 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:14:08.211 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:14:08.211 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:14:08.212 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:14:08.212 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:14:08.212 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:14:08.212 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:14:08.212 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:14:08.212 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992047815314, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:14:08.212 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:14:08.212 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:14:08.212 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992047815626, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1536, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 488, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 366, "raw_average_value_size": 73, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725992047, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "f82ad42c-e2af-4e42-94c3-a2129ea9771e", "db_session_id": "4KVRSJGH3A75CSS3Q9KA"}} 2024-09-10T18:14:08.212 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:14:08.212 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992047815954, "job": 1, "event": "recovery_finished"} 2024-09-10T18:14:08.213 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:14:08.213 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x555e48158700 2024-09-10T18:14:08.213 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: DB pointer 0x555e48128000 2024-09-10T18:14:08.213 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:14:08.213 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:14:08.213 INFO:journalctl@ceph.mon.b.smithi133.stdout: ** DB Stats ** 2024-09-10T18:14:08.213 INFO:journalctl@ceph.mon.b.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:14:08.213 INFO:journalctl@ceph.mon.b.smithi133.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:14:08.213 INFO:journalctl@ceph.mon.b.smithi133.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:14:08.213 INFO:journalctl@ceph.mon.b.smithi133.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:14:08.213 INFO:journalctl@ceph.mon.b.smithi133.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:14:08.213 INFO:journalctl@ceph.mon.b.smithi133.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:14:08.213 INFO:journalctl@ceph.mon.b.smithi133.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:14:08.214 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2024-09-10T18:14:08.214 INFO:journalctl@ceph.mon.b.smithi133.stdout: ** Compaction Stats [default] ** 2024-09-10T18:14:08.214 INFO:journalctl@ceph.mon.b.smithi133.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:14:08.214 INFO:journalctl@ceph.mon.b.smithi133.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:14:08.214 INFO:journalctl@ceph.mon.b.smithi133.stdout: L0 1/0 1.50 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:08.214 INFO:journalctl@ceph.mon.b.smithi133.stdout: Sum 1/0 1.50 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:08.214 INFO:journalctl@ceph.mon.b.smithi133.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:08.215 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2024-09-10T18:14:08.215 INFO:journalctl@ceph.mon.b.smithi133.stdout: ** Compaction Stats [default] ** 2024-09-10T18:14:08.215 INFO:journalctl@ceph.mon.b.smithi133.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:14:08.215 INFO:journalctl@ceph.mon.b.smithi133.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:14:08.215 INFO:journalctl@ceph.mon.b.smithi133.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:08.215 INFO:journalctl@ceph.mon.b.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:14:08.215 INFO:journalctl@ceph.mon.b.smithi133.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:14:08.215 INFO:journalctl@ceph.mon.b.smithi133.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:14:08.215 INFO:journalctl@ceph.mon.b.smithi133.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:14:08.215 INFO:journalctl@ceph.mon.b.smithi133.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:14:08.215 INFO:journalctl@ceph.mon.b.smithi133.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:14:08.215 INFO:journalctl@ceph.mon.b.smithi133.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:14:08.215 INFO:journalctl@ceph.mon.b.smithi133.stdout: Interval compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:14:08.216 INFO:journalctl@ceph.mon.b.smithi133.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:14:08.216 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2024-09-10T18:14:08.216 INFO:journalctl@ceph.mon.b.smithi133.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:14:08.216 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2024-09-10T18:14:08.216 INFO:journalctl@ceph.mon.b.smithi133.stdout: ** Compaction Stats [default] ** 2024-09-10T18:14:08.216 INFO:journalctl@ceph.mon.b.smithi133.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:14:08.216 INFO:journalctl@ceph.mon.b.smithi133.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:14:08.216 INFO:journalctl@ceph.mon.b.smithi133.stdout: L0 1/0 1.50 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:08.216 INFO:journalctl@ceph.mon.b.smithi133.stdout: Sum 1/0 1.50 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:08.216 INFO:journalctl@ceph.mon.b.smithi133.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:14:08.216 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2024-09-10T18:14:08.216 INFO:journalctl@ceph.mon.b.smithi133.stdout: ** Compaction Stats [default] ** 2024-09-10T18:14:08.216 INFO:journalctl@ceph.mon.b.smithi133.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:14:08.217 INFO:journalctl@ceph.mon.b.smithi133.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:14:08.217 INFO:journalctl@ceph.mon.b.smithi133.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:08.217 INFO:journalctl@ceph.mon.b.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:14:08.217 INFO:journalctl@ceph.mon.b.smithi133.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:14:08.217 INFO:journalctl@ceph.mon.b.smithi133.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:14:08.217 INFO:journalctl@ceph.mon.b.smithi133.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:14:08.217 INFO:journalctl@ceph.mon.b.smithi133.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:14:08.217 INFO:journalctl@ceph.mon.b.smithi133.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:14:08.217 INFO:journalctl@ceph.mon.b.smithi133.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:14:08.217 INFO:journalctl@ceph.mon.b.smithi133.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:14:08.217 INFO:journalctl@ceph.mon.b.smithi133.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:14:08.217 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2024-09-10T18:14:08.217 INFO:journalctl@ceph.mon.b.smithi133.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:14:08.218 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:14:08.218 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: using public_addrv v1:172.21.15.133:6789/0 2024-09-10T18:14:08.218 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: starting mon.b rank -1 at public addrs v1:172.21.15.133:6789/0 at bind addrs v1:172.21.15.133:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:14:08.218 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(???) e0 preinit fsid e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:14:08.218 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T18:14:08.218 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T18:14:08.218 INFO:journalctl@ceph.mon.b.smithi133.stdout: e1 2024-09-10T18:14:08.218 INFO:journalctl@ceph.mon.b.smithi133.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:14:08.218 INFO:journalctl@ceph.mon.b.smithi133.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:14:08.218 INFO:journalctl@ceph.mon.b.smithi133.stdout: legacy client fscid: -1 2024-09-10T18:14:08.218 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2024-09-10T18:14:08.218 INFO:journalctl@ceph.mon.b.smithi133.stdout: No filesystems configured 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:08.219 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:08.220 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:08.220 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:14:08.220 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:08.220 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:08.220 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T18:14:08.220 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:14:08.220 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:14:08.220 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:14:08.220 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:08.220 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:08.220 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:08.220 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/619105202' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:14:08.220 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:08.221 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/619105202' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:14:08.221 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:14:08.221 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:08.221 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:08.221 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:08.221 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:08.221 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:08.221 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:08.221 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:08.221 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:08.221 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: from='client.14182 v1:172.21.15.133:0/4097790779' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:[v1:172.21.15.99:6789]=a;smithi099:[v1:172.21.15.99:6790]=c;smithi133:[v1:172.21.15.133:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:14:08.221 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: Saving service mon spec with placement smithi099:[v1:172.21.15.99:6789]=a;smithi099:[v1:172.21.15.99:6790]=c;smithi133:[v1:172.21.15.133:6789]=b;count:3 2024-09-10T18:14:08.221 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: Deploying daemon mon.b on smithi133 2024-09-10T18:14:08.222 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:08.222 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:08.222 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 ceph-mon[31214]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T18:14:08.222 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 bash[31182]: dc0f0783b0838c574c61cb772c281c9f48525a08266ee53c617decc3dec5c730 2024-09-10T18:14:08.222 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:07 smithi133 systemd[1]: Started Ceph mon.b for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:14:08.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:07 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/2472410102' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:14:09.224 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:14:09.225 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:14:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:14:13.082 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:14:13.082 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":2,"fsid":"e9807d90-6f9f-11ef-bcea-c7b262605968","modified":"2024-09-10T18:14:07.837455Z","created":"2024-09-10T18:12:40.967516Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T18:14:13.083 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 2 2024-09-10T18:14:13.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/2472410102' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:14:13.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:13.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:14:13.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:13.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: mon.a calling monitor election 2024-09-10T18:14:13.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:13.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:13.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:13.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: mon.b calling monitor election 2024-09-10T18:14:13.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:13.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:13.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:13.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:13.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:14:13.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: monmap e2: 2 mons at {a=v1:172.21.15.99:6789/0,b=v1:172.21.15.133:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:14:13.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: fsmap 2024-09-10T18:14:13.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:14:13.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: mgrmap e13: y(active, since 39s) 2024-09-10T18:14:13.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: overall HEALTH_OK 2024-09-10T18:14:13.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:13.243 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:13.243 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:13.243 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:13.243 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:13.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:13.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:14:13.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:13.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: mon.a calling monitor election 2024-09-10T18:14:13.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:13.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:13.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:13.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: mon.b calling monitor election 2024-09-10T18:14:13.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:13.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:13.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:13.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:13.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:14:13.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: monmap e2: 2 mons at {a=v1:172.21.15.99:6789/0,b=v1:172.21.15.133:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:14:13.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: fsmap 2024-09-10T18:14:13.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:14:13.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: mgrmap e13: y(active, since 39s) 2024-09-10T18:14:13.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: overall HEALTH_OK 2024-09-10T18:14:13.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:13.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:13.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:13.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:13.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:14.217 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:13 smithi099 ceph-mon[27037]: Deploying daemon mon.c on smithi099 2024-09-10T18:14:14.217 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:13 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/3032930811' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:14:14.217 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:13 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:14.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:13 smithi133 ceph-mon[31214]: Deploying daemon mon.c on smithi099 2024-09-10T18:14:14.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:13 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/3032930811' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:14:14.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:13 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:14.647 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:14:14.647 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:14:14.891 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:14:14.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:14 smithi133 ceph-mon[31214]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:15.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:14 smithi099 ceph-mon[27037]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:16.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:15 smithi099 podman[37538]: 2024-09-10 18:14:15.890393371 +0000 UTC m=+0.133738290 container create 0ea6af5bbd90e3f8f9c8c2ba64b010eaca9444a98e2498e53ef3d6b508fd58ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T18:14:16.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 podman[37538]: 2024-09-10 18:14:16.143155144 +0000 UTC m=+0.386500065 container init 0ea6af5bbd90e3f8f9c8c2ba64b010eaca9444a98e2498e53ef3d6b508fd58ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T18:14:16.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 podman[37538]: 2024-09-10 18:14:16.146543024 +0000 UTC m=+0.389887943 container start 0ea6af5bbd90e3f8f9c8c2ba64b010eaca9444a98e2498e53ef3d6b508fd58ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:14:16.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:14:16.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 2 2024-09-10T18:14:16.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: pidfile_write: ignore empty --pid-file 2024-09-10T18:14:16.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: load: jerasure load: lrc 2024-09-10T18:14:16.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:14:16.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:14:16.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:14:16.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: DB SUMMARY 2024-09-10T18:14:16.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: DB Session ID: KOG6LOUFX38FRMMUWZLB 2024-09-10T18:14:16.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:14:16.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:14:16.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:14:16.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T18:14:16.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 566 ; 2024-09-10T18:14:16.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:14:16.508 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:14:16.508 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:14:16.508 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:14:16.508 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.env: 0x561a69269440 2024-09-10T18:14:16.508 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.fs: Posix File System 2024-09-10T18:14:16.508 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.info_log: 0x561a6a238e80 2024-09-10T18:14:16.508 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:14:16.508 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.statistics: (nil) 2024-09-10T18:14:16.509 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.use_fsync: 0 2024-09-10T18:14:16.509 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:14:16.509 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:14:16.509 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:14:16.509 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:14:16.509 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:14:16.509 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:14:16.509 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:14:16.510 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:14:16.510 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:14:16.510 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:14:16.510 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:14:16.510 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.db_log_dir: 2024-09-10T18:14:16.510 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-10T18:14:16.510 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:14:16.510 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:14:16.510 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:14:16.511 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:14:16.511 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:14:16.511 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:14:16.511 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:14:16.511 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:14:16.511 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.write_buffer_manager: 0x561a6a2f6840 2024-09-10T18:14:16.511 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:14:16.511 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:14:16.511 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:14:16.512 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:14:16.512 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:14:16.512 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:14:16.512 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:14:16.512 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:14:16.512 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:14:16.512 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.unordered_write: 0 2024-09-10T18:14:16.512 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:14:16.512 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:14:16.513 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:14:16.513 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:14:16.513 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.row_cache: None 2024-09-10T18:14:16.513 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.wal_filter: None 2024-09-10T18:14:16.515 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:14:16.515 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:14:16.515 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:14:16.515 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:14:16.515 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:14:16.516 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:14:16.516 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:14:16.516 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:14:16.516 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:14:16.516 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:14:16.516 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:14:16.516 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:14:16.516 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:14:16.517 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:14:16.517 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:14:16.517 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:14:16.517 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:14:16.517 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:14:16.517 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:14:16.517 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:14:16.517 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:14:16.517 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:14:16.518 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:14:16.518 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:14:16.518 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:14:16.518 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:14:16.518 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:14:16.518 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_open_files: -1 2024-09-10T18:14:16.518 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:14:16.518 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:14:16.518 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:14:16.519 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:14:16.519 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:14:16.519 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Compression algorithms supported: 2024-09-10T18:14:16.519 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: kZSTD supported: 0 2024-09-10T18:14:16.519 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:14:16.519 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:14:16.519 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:14:16.519 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:14:16.519 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:14:16.519 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:14:16.520 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:14:16.520 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:14:16.520 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-10T18:14:16.520 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:14:16.520 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:14:16.520 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.merge_operator: 2024-09-10T18:14:16.520 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compaction_filter: None 2024-09-10T18:14:16.520 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:14:16.520 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:14:16.520 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:14:16.520 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:14:16.520 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561a6a1dfa38) 2024-09-10T18:14:16.521 INFO:journalctl@ceph.mon.c.smithi099.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:14:16.521 INFO:journalctl@ceph.mon.c.smithi099.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:14:16.521 INFO:journalctl@ceph.mon.c.smithi099.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:14:16.521 INFO:journalctl@ceph.mon.c.smithi099.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:14:16.521 INFO:journalctl@ceph.mon.c.smithi099.stdout: index_type: 0 2024-09-10T18:14:16.521 INFO:journalctl@ceph.mon.c.smithi099.stdout: data_block_index_type: 0 2024-09-10T18:14:16.521 INFO:journalctl@ceph.mon.c.smithi099.stdout: index_shortening: 1 2024-09-10T18:14:16.521 INFO:journalctl@ceph.mon.c.smithi099.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:14:16.521 INFO:journalctl@ceph.mon.c.smithi099.stdout: hash_index_allow_collision: 1 2024-09-10T18:14:16.521 INFO:journalctl@ceph.mon.c.smithi099.stdout: checksum: 1 2024-09-10T18:14:16.521 INFO:journalctl@ceph.mon.c.smithi099.stdout: no_block_cache: 0 2024-09-10T18:14:16.521 INFO:journalctl@ceph.mon.c.smithi099.stdout: block_cache: 0x561a6a2151f0 2024-09-10T18:14:16.522 INFO:journalctl@ceph.mon.c.smithi099.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:14:16.522 INFO:journalctl@ceph.mon.c.smithi099.stdout: block_cache_options: 2024-09-10T18:14:16.522 INFO:journalctl@ceph.mon.c.smithi099.stdout: capacity : 536870912 2024-09-10T18:14:16.522 INFO:journalctl@ceph.mon.c.smithi099.stdout: num_shard_bits : 4 2024-09-10T18:14:16.522 INFO:journalctl@ceph.mon.c.smithi099.stdout: strict_capacity_limit : 0 2024-09-10T18:14:16.522 INFO:journalctl@ceph.mon.c.smithi099.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:14:16.522 INFO:journalctl@ceph.mon.c.smithi099.stdout: block_cache_compressed: (nil) 2024-09-10T18:14:16.522 INFO:journalctl@ceph.mon.c.smithi099.stdout: persistent_cache: (nil) 2024-09-10T18:14:16.522 INFO:journalctl@ceph.mon.c.smithi099.stdout: block_size: 4096 2024-09-10T18:14:16.522 INFO:journalctl@ceph.mon.c.smithi099.stdout: block_size_deviation: 10 2024-09-10T18:14:16.522 INFO:journalctl@ceph.mon.c.smithi099.stdout: block_restart_interval: 16 2024-09-10T18:14:16.522 INFO:journalctl@ceph.mon.c.smithi099.stdout: index_block_restart_interval: 1 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout: metadata_block_size: 4096 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout: partition_filters: 0 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout: use_delta_encoding: 1 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout: whole_key_filtering: 1 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout: verify_compression: 0 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout: format_version: 4 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout: enable_index_compression: 1 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout: block_align: 0 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compression: NoCompression 2024-09-10T18:14:16.523 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:14:16.524 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:14:16.524 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:14:16.524 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.num_levels: 7 2024-09-10T18:14:16.524 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:14:16.524 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:14:16.524 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:14:16.524 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:14:16.524 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:14:16.524 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:14:16.524 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:14:16.524 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:14:16.524 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:14:16.525 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:14:16.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:14:16.527 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:14:16.527 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:14:16.527 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:14:16.527 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:14:16.527 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:14:16.527 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:14:16.527 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:14:16.527 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:14:16.527 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:14:16.527 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:14:16.527 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:14:16.527 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:14:16.527 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:14:16.528 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:14:16.528 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:14:16.528 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:14:16.528 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:14:16.528 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:14:16.528 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:14:16.528 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:14:16.528 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:14:16.528 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:14:16.528 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:14:16.528 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:14:16.528 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.ttl: 2592000 2024-09-10T18:14:16.528 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:14:16.529 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:14:16.529 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:14:16.529 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:14:16.529 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:14:16.529 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:14:16.529 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:14:16.529 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:14:16.529 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:14:16.529 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:14:16.529 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992056170189, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:14:16.529 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:14:16.529 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:14:16.529 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992056170477, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1626, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 578, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 456, "raw_average_value_size": 91, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725992056, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "97245e31-237e-440e-ad09-90198c507646", "db_session_id": "KOG6LOUFX38FRMMUWZLB"}} 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725992056170786, "job": 1, "event": "recovery_finished"} 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x561a6a248700 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: DB pointer 0x561a6a218000 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout: ** DB Stats ** 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:14:16.530 INFO:journalctl@ceph.mon.c.smithi099.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:14:16.531 INFO:journalctl@ceph.mon.c.smithi099.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:14:16.531 INFO:journalctl@ceph.mon.c.smithi099.stdout: 2024-09-10T18:14:16.531 INFO:journalctl@ceph.mon.c.smithi099.stdout: ** Compaction Stats [default] ** 2024-09-10T18:14:16.531 INFO:journalctl@ceph.mon.c.smithi099.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:14:16.531 INFO:journalctl@ceph.mon.c.smithi099.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:14:16.531 INFO:journalctl@ceph.mon.c.smithi099.stdout: L0 1/0 1.59 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:16.531 INFO:journalctl@ceph.mon.c.smithi099.stdout: Sum 1/0 1.59 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:16.531 INFO:journalctl@ceph.mon.c.smithi099.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:16.531 INFO:journalctl@ceph.mon.c.smithi099.stdout: 2024-09-10T18:14:16.531 INFO:journalctl@ceph.mon.c.smithi099.stdout: ** Compaction Stats [default] ** 2024-09-10T18:14:16.531 INFO:journalctl@ceph.mon.c.smithi099.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:14:16.531 INFO:journalctl@ceph.mon.c.smithi099.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:14:16.531 INFO:journalctl@ceph.mon.c.smithi099.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:16.532 INFO:journalctl@ceph.mon.c.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:14:16.532 INFO:journalctl@ceph.mon.c.smithi099.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:14:16.532 INFO:journalctl@ceph.mon.c.smithi099.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:14:16.532 INFO:journalctl@ceph.mon.c.smithi099.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:14:16.532 INFO:journalctl@ceph.mon.c.smithi099.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:14:16.532 INFO:journalctl@ceph.mon.c.smithi099.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:14:16.532 INFO:journalctl@ceph.mon.c.smithi099.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:14:16.532 INFO:journalctl@ceph.mon.c.smithi099.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:14:16.532 INFO:journalctl@ceph.mon.c.smithi099.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:14:16.532 INFO:journalctl@ceph.mon.c.smithi099.stdout: 2024-09-10T18:14:16.532 INFO:journalctl@ceph.mon.c.smithi099.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:14:16.532 INFO:journalctl@ceph.mon.c.smithi099.stdout: 2024-09-10T18:14:16.532 INFO:journalctl@ceph.mon.c.smithi099.stdout: ** Compaction Stats [default] ** 2024-09-10T18:14:16.533 INFO:journalctl@ceph.mon.c.smithi099.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:14:16.533 INFO:journalctl@ceph.mon.c.smithi099.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:14:16.533 INFO:journalctl@ceph.mon.c.smithi099.stdout: L0 1/0 1.59 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:16.533 INFO:journalctl@ceph.mon.c.smithi099.stdout: Sum 1/0 1.59 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:16.533 INFO:journalctl@ceph.mon.c.smithi099.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:14:16.533 INFO:journalctl@ceph.mon.c.smithi099.stdout: 2024-09-10T18:14:16.533 INFO:journalctl@ceph.mon.c.smithi099.stdout: ** Compaction Stats [default] ** 2024-09-10T18:14:16.533 INFO:journalctl@ceph.mon.c.smithi099.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:14:16.533 INFO:journalctl@ceph.mon.c.smithi099.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:14:16.533 INFO:journalctl@ceph.mon.c.smithi099.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-10T18:14:16.533 INFO:journalctl@ceph.mon.c.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:14:16.533 INFO:journalctl@ceph.mon.c.smithi099.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:14:16.533 INFO:journalctl@ceph.mon.c.smithi099.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:14:16.534 INFO:journalctl@ceph.mon.c.smithi099.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:14:16.534 INFO:journalctl@ceph.mon.c.smithi099.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:14:16.534 INFO:journalctl@ceph.mon.c.smithi099.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:14:16.534 INFO:journalctl@ceph.mon.c.smithi099.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:14:16.534 INFO:journalctl@ceph.mon.c.smithi099.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:14:16.534 INFO:journalctl@ceph.mon.c.smithi099.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:14:16.534 INFO:journalctl@ceph.mon.c.smithi099.stdout: 2024-09-10T18:14:16.534 INFO:journalctl@ceph.mon.c.smithi099.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:14:16.534 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:14:16.534 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: using public_addrv v1:172.21.15.99:6790/0 2024-09-10T18:14:16.534 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: starting mon.c rank -1 at public addrs v1:172.21.15.99:6790/0 at bind addrs v1:172.21.15.99:6790/0 mon_data /var/lib/ceph/mon/ceph-c fsid e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:14:16.534 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(???) e0 preinit fsid e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:14:16.534 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout: e1 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout: legacy client fscid: -1 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout: 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout: No filesystems configured 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:14:16.535 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:14:16.536 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:14:16.536 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:14:16.536 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:14:16.536 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:16.536 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:16.536 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:16.536 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:16.536 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:16.536 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:14:16.536 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:16.536 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:16.536 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T18:14:16.536 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:14:16.537 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:14:16.537 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:14:16.537 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:16.537 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:16.537 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:16.537 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/619105202' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:14:16.537 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:16.537 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/619105202' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:14:16.537 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:14:16.537 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:16.537 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:16.537 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:16.537 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:16.538 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:16.538 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:16.538 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:16.538 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:16.538 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='client.14182 v1:172.21.15.133:0/4097790779' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:[v1:172.21.15.99:6789]=a;smithi099:[v1:172.21.15.99:6790]=c;smithi133:[v1:172.21.15.133:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:14:16.538 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: Saving service mon spec with placement smithi099:[v1:172.21.15.99:6789]=a;smithi099:[v1:172.21.15.99:6790]=c;smithi133:[v1:172.21.15.133:6789]=b;count:3 2024-09-10T18:14:16.538 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: Deploying daemon mon.b on smithi133 2024-09-10T18:14:16.538 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:16.538 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:16.538 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/2472410102' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:14:16.538 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:16.538 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:14:16.539 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:16.539 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.a calling monitor election 2024-09-10T18:14:16.539 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:16.539 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:16.539 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:16.539 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.b calling monitor election 2024-09-10T18:14:16.539 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:16.539 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:16.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:16.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:16.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:14:16.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: monmap e2: 2 mons at {a=v1:172.21.15.99:6789/0,b=v1:172.21.15.133:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:14:16.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: fsmap 2024-09-10T18:14:16.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:14:16.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mgrmap e13: y(active, since 39s) 2024-09-10T18:14:16.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: overall HEALTH_OK 2024-09-10T18:14:16.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:16.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:16.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:16.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:16.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:16.541 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: Deploying daemon mon.c on smithi099 2024-09-10T18:14:16.541 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/3032930811' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:14:16.541 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:16.541 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:16.541 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 ceph-mon[37557]: mon.c@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-09-10T18:14:16.541 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 bash[37538]: 0ea6af5bbd90e3f8f9c8c2ba64b010eaca9444a98e2498e53ef3d6b508fd58ad 2024-09-10T18:14:16.541 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:16 smithi099 systemd[1]: Started Ceph mon.c for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:14:21.452 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T18:14:21.452 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":3,"fsid":"e9807d90-6f9f-11ef-bcea-c7b262605968","modified":"2024-09-10T18:14:16.190369Z","created":"2024-09-10T18:12:40.967516Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6790","nonce":0}]},"addr":"172.21.15.99:6790/0","public_addr":"172.21.15.99:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-10T18:14:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 3 2024-09-10T18:14:21.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:21.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:14:21.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: mon.a calling monitor election 2024-09-10T18:14:21.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: mon.b calling monitor election 2024-09-10T18:14:21.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:21.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:21.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: mon.c calling monitor election 2024-09-10T18:14:21.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:21.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:14:21.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: monmap e3: 3 mons at {a=v1:172.21.15.99:6789/0,b=v1:172.21.15.133:6789/0,c=v1:172.21.15.99:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:14:21.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: fsmap 2024-09-10T18:14:21.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:14:21.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: mgrmap e13: y(active, since 47s) 2024-09-10T18:14:21.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: overall HEALTH_OK 2024-09-10T18:14:21.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:21.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:21.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:21.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:21.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:21.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:14:21.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: mon.a calling monitor election 2024-09-10T18:14:21.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: mon.b calling monitor election 2024-09-10T18:14:21.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:21.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:21.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: mon.c calling monitor election 2024-09-10T18:14:21.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:21.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:14:21.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: monmap e3: 3 mons at {a=v1:172.21.15.99:6789/0,b=v1:172.21.15.133:6789/0,c=v1:172.21.15.99:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:14:21.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: fsmap 2024-09-10T18:14:21.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:14:21.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: mgrmap e13: y(active, since 47s) 2024-09-10T18:14:21.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: overall HEALTH_OK 2024-09-10T18:14:21.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:21.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:21.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:21.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:21.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:21.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:14:21.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: mon.a calling monitor election 2024-09-10T18:14:21.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: mon.b calling monitor election 2024-09-10T18:14:21.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:14:21.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:21.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: mon.c calling monitor election 2024-09-10T18:14:21.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:21.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:21.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:14:21.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: monmap e3: 3 mons at {a=v1:172.21.15.99:6789/0,b=v1:172.21.15.133:6789/0,c=v1:172.21.15.99:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:14:21.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: fsmap 2024-09-10T18:14:21.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:14:21.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: mgrmap e13: y(active, since 47s) 2024-09-10T18:14:21.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: overall HEALTH_OK 2024-09-10T18:14:21.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:21.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:21.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:21.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:22.101 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T18:14:22.101 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T18:14:22.360 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:14:22.566 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:22 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:22.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:22 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/555258096' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:14:22.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:22 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:22.608 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:22 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:22.608 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:22 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/555258096' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:14:22.608 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:22 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:22.609 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:22 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:22.609 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:22 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/555258096' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:14:22.609 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:22 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:14:23.663 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:23.663 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.664 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.664 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.664 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.664 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:23.664 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:23.664 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.664 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.664 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.664 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.664 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.664 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:23.664 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:14:23.664 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:23.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:23.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:23.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:23.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.665 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:23.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:14:23.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:23.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:14:23.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:24.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[27037]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-10T18:14:24.717 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[27037]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T18:14:24.717 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[27037]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:14:24.717 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[27037]: Updating smithi099:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:14:24.717 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[27037]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:14:24.717 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[27037]: Reconfiguring daemon mon.a on smithi099 2024-09-10T18:14:24.717 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:24.718 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[37557]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-10T18:14:24.718 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[37557]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T18:14:24.718 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[37557]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:14:24.718 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[37557]: Updating smithi099:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:14:24.718 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[37557]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:14:24.718 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[37557]: Reconfiguring daemon mon.a on smithi099 2024-09-10T18:14:24.718 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:24 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:24.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:24 smithi133 ceph-mon[31214]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-10T18:14:24.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:24 smithi133 ceph-mon[31214]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T18:14:24.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:24 smithi133 ceph-mon[31214]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:14:24.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:24 smithi133 ceph-mon[31214]: Updating smithi099:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:14:24.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:24 smithi133 ceph-mon[31214]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:14:24.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:24 smithi133 ceph-mon[31214]: Reconfiguring daemon mon.a on smithi099 2024-09-10T18:14:24.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:24 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:25.093 INFO:teuthology.orchestra.run.smithi099.stdout:# minimal ceph.conf for e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:14:25.093 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2024-09-10T18:14:25.093 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = e9807d90-6f9f-11ef-bcea-c7b262605968 2024-09-10T18:14:25.093 INFO:teuthology.orchestra.run.smithi099.stdout: mon_host = 172.21.15.99:6789/0 172.21.15.133:6789/0 v1:172.21.15.99:6790/0 2024-09-10T18:14:25.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:25 smithi133 ceph-mon[31214]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:25.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:25 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:25.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:25 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:25.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:25 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:25.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:25 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:14:25.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:25 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:25.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:25 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3743683548' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:25.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[27037]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:25.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:25.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:25.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:25.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:14:25.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:25.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3743683548' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:25.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[37557]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:25.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:25.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:25.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:25.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:14:25.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:25.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:25 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3743683548' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:25.820 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T18:14:25.820 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:14:25.820 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:14:25.848 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:14:25.848 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:14:25.913 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T18:14:25.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:14:25.944 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T18:14:25.944 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:14:26.011 INFO:tasks.cephadm:Adding mgr.y on smithi099 2024-09-10T18:14:26.011 INFO:tasks.cephadm:Adding mgr.x on smithi133 2024-09-10T18:14:26.012 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi099=y;smithi133=x' 2024-09-10T18:14:26.284 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:14:26.636 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:26 smithi099 ceph-mon[27037]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:14:26.636 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:26 smithi099 ceph-mon[27037]: Reconfiguring daemon mon.c on smithi099 2024-09-10T18:14:26.636 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:26 smithi099 ceph-mon[37557]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:14:26.636 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:26 smithi099 ceph-mon[37557]: Reconfiguring daemon mon.c on smithi099 2024-09-10T18:14:26.678 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:26 smithi133 ceph-mon[31214]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:14:26.678 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:26 smithi133 ceph-mon[31214]: Reconfiguring daemon mon.c on smithi099 2024-09-10T18:14:27.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:27 smithi133 ceph-mon[31214]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:27.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:27 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:27.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:27 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:27.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:27 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:27.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:27 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:14:27.568 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:27 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:27.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:27 smithi099 ceph-mon[27037]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:27.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:27 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:27.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:27 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:27.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:27 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:27.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:27 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:14:27.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:27 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:27.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:27 smithi099 ceph-mon[37557]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:27.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:27 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:27.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:27 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:27.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:27 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:14:27.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:27 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:14:27.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:27 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:28.706 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mgr update... 2024-09-10T18:14:28.706 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:28 smithi133 ceph-mon[31214]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:14:28.706 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:28 smithi133 ceph-mon[31214]: Reconfiguring daemon mon.b on smithi133 2024-09-10T18:14:28.707 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:28.707 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:28.707 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:28.707 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:28.707 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:28.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[27037]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:14:28.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[27037]: Reconfiguring daemon mon.b on smithi133 2024-09-10T18:14:28.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:28.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:28.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:28.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:28.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:28.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[37557]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:14:28.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[37557]: Reconfiguring daemon mon.b on smithi133 2024-09-10T18:14:28.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:28.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:28.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:28.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:28.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:29.259 DEBUG:teuthology.orchestra.run.smithi133:mgr.x> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T18:14:29.261 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T18:14:29.261 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:14:29.261 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:14:29.281 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:14:29.281 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-09-10T18:14:29.340 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:14:29.341 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:14:29.341 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 970 Links: 1 2024-09-10T18:14:29.341 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:14:29.341 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:14:29.341 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-10 18:14:15.602383455 +0000 2024-09-10T18:14:29.341 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-10 18:08:34.654465720 +0000 2024-09-10T18:14:29.341 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-10 18:08:34.654465720 +0000 2024-09-10T18:14:29.341 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-10 18:08:34.654465720 +0000 2024-09-10T18:14:29.342 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:14:29.409 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-10T18:14:29.409 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-10T18:14:29.409 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000236808 s, 2.2 MB/s 2024-09-10T18:14:29.410 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:14:29.469 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-09-10T18:14:29.526 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:14:29.526 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:14:29.527 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-10T18:14:29.527 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:14:29.527 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:14:29.527 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-10 18:14:15.602383455 +0000 2024-09-10T18:14:29.527 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-10 18:08:34.654465720 +0000 2024-09-10T18:14:29.527 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-10 18:08:34.654465720 +0000 2024-09-10T18:14:29.527 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-10 18:08:34.654465720 +0000 2024-09-10T18:14:29.528 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:14:29.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:29 smithi133 ceph-mon[31214]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:29.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:29.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:29.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:29.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:29.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:29.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:29.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:14:29.552 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:14:29.553 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:14:29.553 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:29.592 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-10T18:14:29.592 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-10T18:14:29.592 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000229955 s, 2.2 MB/s 2024-09-10T18:14:29.593 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:14:29.651 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-09-10T18:14:29.707 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:14:29.708 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:14:29.708 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 977 Links: 1 2024-09-10T18:14:29.708 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:14:29.708 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:14:29.708 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-10 18:14:15.603383431 +0000 2024-09-10T18:14:29.708 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-10 18:08:34.655465564 +0000 2024-09-10T18:14:29.708 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-10 18:08:34.655465564 +0000 2024-09-10T18:14:29.708 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-10 18:08:34.655465564 +0000 2024-09-10T18:14:29.709 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:14:29.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[37557]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:29.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:29.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:29.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:29.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:29.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:29.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:29.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:14:29.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:14:29.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:14:29.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:29.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[27037]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:29.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:29.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:29.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:29.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:29.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:29.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:29.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:14:29.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:14:29.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:14:29.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:29.772 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-10T18:14:29.772 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-10T18:14:29.772 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000290975 s, 1.8 MB/s 2024-09-10T18:14:29.774 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:14:29.830 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-09-10T18:14:29.886 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:14:29.887 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:14:29.887 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 983 Links: 1 2024-09-10T18:14:29.887 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:14:29.887 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:14:29.887 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-10 18:14:15.603383431 +0000 2024-09-10T18:14:29.888 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-10 18:08:34.655465564 +0000 2024-09-10T18:14:29.888 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-10 18:08:34.655465564 +0000 2024-09-10T18:14:29.888 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-10 18:08:34.655465564 +0000 2024-09-10T18:14:29.888 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:14:29.950 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-10T18:14:29.950 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-10T18:14:29.950 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000312798 s, 1.6 MB/s 2024-09-10T18:14:29.951 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:14:30.008 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T18:14:30.008 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:14:30.024 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:14:30.024 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-09-10T18:14:30.081 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:14:30.081 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:14:30.081 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 979 Links: 1 2024-09-10T18:14:30.081 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:14:30.081 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:14:30.081 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-10 18:14:07.119373322 +0000 2024-09-10T18:14:30.081 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-10 18:08:34.561874635 +0000 2024-09-10T18:14:30.081 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-10 18:08:34.561874635 +0000 2024-09-10T18:14:30.081 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-10 18:08:34.561874635 +0000 2024-09-10T18:14:30.082 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:14:30.145 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-10T18:14:30.145 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-10T18:14:30.145 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000207774 s, 2.5 MB/s 2024-09-10T18:14:30.146 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:14:30.203 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-09-10T18:14:30.259 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:14:30.259 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:14:30.259 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-10T18:14:30.259 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:14:30.259 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:14:30.259 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-10 18:14:07.119373322 +0000 2024-09-10T18:14:30.259 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-10 18:08:34.560874731 +0000 2024-09-10T18:14:30.259 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-10 18:08:34.560874731 +0000 2024-09-10T18:14:30.259 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-10 18:08:34.560874731 +0000 2024-09-10T18:14:30.260 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:14:30.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-10T18:14:30.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-10T18:14:30.433 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000222704 s, 2.3 MB/s 2024-09-10T18:14:30.434 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:14:30.452 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-09-10T18:14:30.514 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:14:30.515 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:14:30.515 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 983 Links: 1 2024-09-10T18:14:30.515 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:14:30.515 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:14:30.516 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-10 18:14:30.386922270 +0000 2024-09-10T18:14:30.516 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-10 18:08:34.561874635 +0000 2024-09-10T18:14:30.516 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-10 18:08:34.561874635 +0000 2024-09-10T18:14:30.516 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-10 18:08:34.561874635 +0000 2024-09-10T18:14:30.516 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:14:30.621 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-10T18:14:30.621 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-10T18:14:30.621 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000253395 s, 2.0 MB/s 2024-09-10T18:14:30.622 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:14:30.638 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-09-10T18:14:30.695 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:14:30.696 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:14:30.696 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-10T18:14:30.696 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:14:30.696 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:14:30.696 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-10 18:14:30.574918626 +0000 2024-09-10T18:14:30.696 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-10 18:08:34.560874731 +0000 2024-09-10T18:14:30.696 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-10 18:08:34.560874731 +0000 2024-09-10T18:14:30.697 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-10 18:08:34.560874731 +0000 2024-09-10T18:14:30.697 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:14:30.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:30 smithi133 ceph-mon[31214]: from='client.24109 v1:172.21.15.133:0/1466469990' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi099=y;smithi133=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:14:30.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:30 smithi133 ceph-mon[31214]: Saving service mgr spec with placement smithi099=y;smithi133=x;count:2 2024-09-10T18:14:30.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:30 smithi133 ceph-mon[31214]: Deploying daemon mgr.x on smithi133 2024-09-10T18:14:30.741 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:30 smithi133 systemd[1]: Starting Ceph mgr.x for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:14:30.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:30 smithi099 ceph-mon[27037]: from='client.24109 v1:172.21.15.133:0/1466469990' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi099=y;smithi133=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:14:30.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:30 smithi099 ceph-mon[27037]: Saving service mgr spec with placement smithi099=y;smithi133=x;count:2 2024-09-10T18:14:30.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:30 smithi099 ceph-mon[27037]: Deploying daemon mgr.x on smithi133 2024-09-10T18:14:30.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:30 smithi099 ceph-mon[37557]: from='client.24109 v1:172.21.15.133:0/1466469990' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi099=y;smithi133=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:14:30.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:30 smithi099 ceph-mon[37557]: Saving service mgr spec with placement smithi099=y;smithi133=x;count:2 2024-09-10T18:14:30.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:30 smithi099 ceph-mon[37557]: Deploying daemon mgr.x on smithi133 2024-09-10T18:14:30.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-10T18:14:30.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-10T18:14:30.761 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.00017039 s, 3.0 MB/s 2024-09-10T18:14:30.763 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:14:30.821 INFO:tasks.cephadm:Deploying osd.0 on smithi099 with /dev/vg_nvme/lv_4... 2024-09-10T18:14:30.821 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:14:31.020 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:30 smithi133 podman[33036]: 2024-09-10 18:14:30.850493243 +0000 UTC m=+0.112687821 container create 4c566f03fa35122fbaf31ea92ddd4265c8b5e5b740740a022d6e0ac0171c73f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T18:14:31.021 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:30 smithi133 podman[33036]: 2024-09-10 18:14:30.754173563 +0000 UTC m=+0.016368109 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:14:31.021 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:31 smithi133 podman[33036]: 2024-09-10 18:14:31.019376755 +0000 UTC m=+0.281571297 container init 4c566f03fa35122fbaf31ea92ddd4265c8b5e5b740740a022d6e0ac0171c73f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T18:14:31.067 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:14:31.392 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:31 smithi133 podman[33036]: 2024-09-10 18:14:31.022096335 +0000 UTC m=+0.284290868 container start 4c566f03fa35122fbaf31ea92ddd4265c8b5e5b740740a022d6e0ac0171c73f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:14:31.392 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:31 smithi133 bash[33036]: 4c566f03fa35122fbaf31ea92ddd4265c8b5e5b740740a022d6e0ac0171c73f6 2024-09-10T18:14:31.392 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:31 smithi133 systemd[1]: Started Ceph mgr.x for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:14:31.392 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:31 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:31.133+0000 7f0a762b9ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:14:31.392 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:31 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:31.192+0000 7f0a762b9ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:14:31.392 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:31 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:31.259+0000 7f0a762b9ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:14:31.392 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:31 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:31.389+0000 7f0a762b9ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:14:31.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:31 smithi133 ceph-mon[31214]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:31.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:31 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:31.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:31 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:31.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:31 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:31.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:31 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:31.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:31 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:31.741 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:31 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:31.538+0000 7f0a762b9ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:14:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:31 smithi099 ceph-mon[27037]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:31 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:31 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:31 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:31 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:31 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:31.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:31 smithi099 ceph-mon[37557]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:31 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:31 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:31 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:31 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:31 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:32.240 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:31 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:31.810+0000 7f0a762b9ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:14:32.241 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:31 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:31.873+0000 7f0a762b9ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:14:32.241 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:32 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:32.041+0000 7f0a762b9ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:14:32.789 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:14:32.882 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:14:32.897 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-09-10T18:14:32.897 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-09-10T18:14:32.897 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134086 s, 782 MB/s 2024-09-10T18:14:32.897 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-09-10T18:14:33.240 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:32 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:32.892+0000 7f0a762b9ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:14:33.240 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:32 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:32.955+0000 7f0a762b9ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:14:33.657 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-mon[31214]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:33.657 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:33.658 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:33.658 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:33.658 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:33.658 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:33.658 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-mon[31214]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:14:33.658 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:14:33.658 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:14:33.658 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:33.658 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-mon[31214]: Reconfiguring daemon mgr.y on smithi099 2024-09-10T18:14:33.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[27037]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:33.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:33.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:33.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:33.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:33.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:33.670 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[27037]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:14:33.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:14:33.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:14:33.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:33.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[27037]: Reconfiguring daemon mgr.y on smithi099 2024-09-10T18:14:33.671 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[37557]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:33.671 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:33.671 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:33.671 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:33.671 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:33.671 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:33.672 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[37557]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:14:33.672 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:14:33.672 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:14:33.672 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:33.672 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:33 smithi099 ceph-mon[37557]: Reconfiguring daemon mgr.y on smithi099 2024-09-10T18:14:33.945 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:33.655+0000 7f0a762b9ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:14:33.945 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:33.765+0000 7f0a762b9ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:14:33.945 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:33.884+0000 7f0a762b9ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:14:33.971 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_4 2024-09-10T18:14:34.241 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:33 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:33.943+0000 7f0a762b9ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:14:34.241 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:34.067+0000 7f0a762b9ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:14:34.241 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:34.133+0000 7f0a762b9ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:14:34.241 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:34.190+0000 7f0a762b9ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:14:34.395 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:14:34.599 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:34.326+0000 7f0a762b9ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:14:34.599 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:34.390+0000 7f0a762b9ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:14:34.600 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:34.457+0000 7f0a762b9ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:14:34.600 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:34.541+0000 7f0a762b9ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:14:34.600 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:34.597+0000 7f0a762b9ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:14:34.990 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:34.654+0000 7f0a762b9ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:14:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-mon[31214]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:34 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[27037]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[37557]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:34 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:35.740 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:14:35 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:14:35.389+0000 7f0a762b9ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:14:36.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:35 smithi099 ceph-mon[37557]: Standby manager daemon x started 2024-09-10T18:14:36.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:35 smithi099 ceph-mon[37557]: from='mgr.? v1:172.21.15.133:0/1660124631' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:14:36.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:35 smithi099 ceph-mon[37557]: from='mgr.? v1:172.21.15.133:0/1660124631' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:14:36.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:35 smithi099 ceph-mon[37557]: from='mgr.? v1:172.21.15.133:0/1660124631' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:14:36.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:35 smithi099 ceph-mon[37557]: from='mgr.? v1:172.21.15.133:0/1660124631' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:14:36.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:35 smithi099 ceph-mon[27037]: Standby manager daemon x started 2024-09-10T18:14:36.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:35 smithi099 ceph-mon[27037]: from='mgr.? v1:172.21.15.133:0/1660124631' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:14:36.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:35 smithi099 ceph-mon[27037]: from='mgr.? v1:172.21.15.133:0/1660124631' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:14:36.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:35 smithi099 ceph-mon[27037]: from='mgr.? v1:172.21.15.133:0/1660124631' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:14:36.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:35 smithi099 ceph-mon[27037]: from='mgr.? v1:172.21.15.133:0/1660124631' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:14:36.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:35 smithi133 ceph-mon[31214]: Standby manager daemon x started 2024-09-10T18:14:36.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:35 smithi133 ceph-mon[31214]: from='mgr.? v1:172.21.15.133:0/1660124631' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:14:36.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:35 smithi133 ceph-mon[31214]: from='mgr.? v1:172.21.15.133:0/1660124631' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:14:36.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:35 smithi133 ceph-mon[31214]: from='mgr.? v1:172.21.15.133:0/1660124631' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:14:36.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:35 smithi133 ceph-mon[31214]: from='mgr.? v1:172.21.15.133:0/1660124631' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:14:37.157 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[37557]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:37.157 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[37557]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-10T18:14:37.157 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:14:37.157 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[37557]: from='client.34115 v1:172.21.15.99:0/3638140796' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:14:37.158 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:14:37.158 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:14:37.158 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:37.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[27037]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:37.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[27037]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-10T18:14:37.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:14:37.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[27037]: from='client.34115 v1:172.21.15.99:0/3638140796' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:14:37.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:14:37.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:14:37.158 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:36 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:37.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:36 smithi133 ceph-mon[31214]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:37.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:36 smithi133 ceph-mon[31214]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-10T18:14:37.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:36 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:14:37.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:36 smithi133 ceph-mon[31214]: from='client.34115 v1:172.21.15.99:0/3638140796' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:14:37.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:36 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:14:37.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:36 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:14:37.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:36 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:39.160 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:38 smithi099 ceph-mon[27037]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:39.160 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:38 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3439267250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d91a1a4-657a-4417-a71c-ac9576241c4c"}]: dispatch 2024-09-10T18:14:39.160 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:38 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3439267250' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d91a1a4-657a-4417-a71c-ac9576241c4c"}]': finished 2024-09-10T18:14:39.160 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:38 smithi099 ceph-mon[27037]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:14:39.161 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:38 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:39.161 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:38 smithi099 ceph-mon[37557]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:39.161 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:38 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3439267250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d91a1a4-657a-4417-a71c-ac9576241c4c"}]: dispatch 2024-09-10T18:14:39.161 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:38 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3439267250' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d91a1a4-657a-4417-a71c-ac9576241c4c"}]': finished 2024-09-10T18:14:39.161 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:38 smithi099 ceph-mon[37557]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:14:39.161 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:38 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:39.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:38 smithi133 ceph-mon[31214]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:39.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:38 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3439267250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d91a1a4-657a-4417-a71c-ac9576241c4c"}]: dispatch 2024-09-10T18:14:39.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:38 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3439267250' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d91a1a4-657a-4417-a71c-ac9576241c4c"}]': finished 2024-09-10T18:14:39.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:38 smithi133 ceph-mon[31214]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:14:39.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:38 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:40.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:39 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3639636661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:14:40.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:39 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3639636661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:14:40.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:39 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3639636661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:14:41.070 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:40 smithi099 ceph-mon[27037]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:41.071 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:40 smithi099 ceph-mon[37557]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:41.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:40 smithi133 ceph-mon[31214]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:43.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:42 smithi099 ceph-mon[27037]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:43.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:42 smithi099 ceph-mon[37557]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:43.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:42 smithi133 ceph-mon[31214]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:44.871 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:44 smithi099 ceph-mon[27037]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:44.872 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:44 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:14:44.872 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:44 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:44.872 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:44 smithi099 ceph-mon[27037]: Deploying daemon osd.0 on smithi099 2024-09-10T18:14:44.872 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:44 smithi099 ceph-mon[37557]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:44.872 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:44 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:14:44.872 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:44 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:44.872 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:44 smithi099 ceph-mon[37557]: Deploying daemon osd.0 on smithi099 2024-09-10T18:14:45.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:44 smithi133 ceph-mon[31214]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:45.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:44 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:14:45.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:44 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:45.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:44 smithi133 ceph-mon[31214]: Deploying daemon osd.0 on smithi099 2024-09-10T18:14:46.955 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:46 smithi099 ceph-mon[27037]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:46.955 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:46 smithi099 ceph-mon[37557]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:47.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:46 smithi133 ceph-mon[31214]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:49.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:48 smithi133 ceph-mon[31214]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:48 smithi099 ceph-mon[27037]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:49.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:48 smithi099 ceph-mon[37557]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:49 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:49 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:49 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:49 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:49 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:49 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:50.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:49 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:50.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:49 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:50.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:49 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:51.236 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:51 smithi099 ceph-mon[27037]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:51.236 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:51 smithi099 ceph-mon[37557]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:51.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:51 smithi133 ceph-mon[31214]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:52.134 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 0 on host 'smithi099' 2024-09-10T18:14:52.684 DEBUG:teuthology.orchestra.run.smithi099:osd.0> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T18:14:52.687 INFO:tasks.cephadm:Deploying osd.1 on smithi099 with /dev/vg_nvme/lv_3... 2024-09-10T18:14:52.687 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:14:52.978 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:14:53.381 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:53 smithi099 ceph-mon[27037]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:53.381 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:53 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:53.381 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:53 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:53.381 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:53 smithi099 ceph-mon[27037]: from='osd.0 v1:172.21.15.99:6801/24904954' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:14:53.381 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:53 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:53.381 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:53 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:53.381 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:53 smithi099 ceph-mon[37557]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:53.382 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:53 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:53.382 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:53 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:53.382 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:53 smithi099 ceph-mon[37557]: from='osd.0 v1:172.21.15.99:6801/24904954' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:14:53.382 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:53 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:53.382 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:53 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:53.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:53 smithi133 ceph-mon[31214]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:53.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:53 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:53.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:53 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:53.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:53 smithi133 ceph-mon[31214]: from='osd.0 v1:172.21.15.99:6801/24904954' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:14:53.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:53 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:53.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:53 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:54.473 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:54 smithi099 ceph-mon[27037]: from='osd.0 v1:172.21.15.99:6801/24904954' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:14:54.473 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:54 smithi099 ceph-mon[27037]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:14:54.473 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:54 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:54.473 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:54 smithi099 ceph-mon[27037]: from='osd.0 v1:172.21.15.99:6801/24904954' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:14:54.474 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:14:54 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0[41432]: 2024-09-10T18:14:54.137+0000 7f109574f640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T18:14:54.474 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:14:54 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0[41432]: 2024-09-10T18:14:54.140+0000 7f109258a640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:14:54.474 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:54 smithi099 ceph-mon[37557]: from='osd.0 v1:172.21.15.99:6801/24904954' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:14:54.474 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:54 smithi099 ceph-mon[37557]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:14:54.474 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:54 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:54.474 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:54 smithi099 ceph-mon[37557]: from='osd.0 v1:172.21.15.99:6801/24904954' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:14:54.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:54 smithi133 ceph-mon[31214]: from='osd.0 v1:172.21.15.99:6801/24904954' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:14:54.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:54 smithi133 ceph-mon[31214]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:14:54.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:54 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:54.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:54 smithi133 ceph-mon[31214]: from='osd.0 v1:172.21.15.99:6801/24904954' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:14:55.422 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:55 smithi099 ceph-mon[37557]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:55.422 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:55 smithi099 ceph-mon[37557]: from='osd.0 v1:172.21.15.99:6801/24904954' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-10T18:14:55.422 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:55 smithi099 ceph-mon[37557]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:14:55.422 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:55 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:55.422 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:55 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:55.422 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:55 smithi099 ceph-mon[27037]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:55.422 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:55 smithi099 ceph-mon[27037]: from='osd.0 v1:172.21.15.99:6801/24904954' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-10T18:14:55.422 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:55 smithi099 ceph-mon[27037]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:14:55.422 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:55 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:55.423 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:55 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:55.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:55 smithi133 ceph-mon[31214]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:55.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:55 smithi133 ceph-mon[31214]: from='osd.0 v1:172.21.15.99:6801/24904954' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-10T18:14:55.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:55 smithi133 ceph-mon[31214]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:14:55.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:55 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:55.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:55 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:56.178 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:14:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:14:56.286 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-09-10T18:14:56.286 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-09-10T18:14:56.286 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125422 s, 836 MB/s 2024-09-10T18:14:56.287 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-09-10T18:14:56.305 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:56 smithi099 ceph-mon[37557]: purged_snaps scrub starts 2024-09-10T18:14:56.305 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:56 smithi099 ceph-mon[37557]: purged_snaps scrub ok 2024-09-10T18:14:56.305 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:56 smithi099 ceph-mon[37557]: osd.0 v1:172.21.15.99:6801/24904954 boot 2024-09-10T18:14:56.305 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:56 smithi099 ceph-mon[37557]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:14:56.305 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:56 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:56.305 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:56 smithi099 ceph-mon[27037]: purged_snaps scrub starts 2024-09-10T18:14:56.305 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:56 smithi099 ceph-mon[27037]: purged_snaps scrub ok 2024-09-10T18:14:56.305 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:56 smithi099 ceph-mon[27037]: osd.0 v1:172.21.15.99:6801/24904954 boot 2024-09-10T18:14:56.306 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:56 smithi099 ceph-mon[27037]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:14:56.306 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:56 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:56.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:56 smithi133 ceph-mon[31214]: purged_snaps scrub starts 2024-09-10T18:14:56.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:56 smithi133 ceph-mon[31214]: purged_snaps scrub ok 2024-09-10T18:14:56.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:56 smithi133 ceph-mon[31214]: osd.0 v1:172.21.15.99:6801/24904954 boot 2024-09-10T18:14:56.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:56 smithi133 ceph-mon[31214]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:14:56.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:56 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:14:57.349 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[27037]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:57.349 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[27037]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:14:57.349 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.349 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.349 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:14:57.349 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.349 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:57.349 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:57.349 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.349 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:57.350 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:57.350 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:57.350 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.350 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[37557]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:57.350 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[37557]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:14:57.350 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.350 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.350 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:14:57.350 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.350 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:57.350 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:57.351 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.351 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:57.351 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:57.351 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:57.351 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:57 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.372 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_3 2024-09-10T18:14:57.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:57 smithi133 ceph-mon[31214]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:14:57.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:57 smithi133 ceph-mon[31214]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:14:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:57 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:57 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:57 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:14:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:57 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:57 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:57 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:57 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:57 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:14:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:57 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:14:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:57 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:14:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:57 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:14:57.610 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:14:58.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:58 smithi133 ceph-mon[31214]: Detected new or changed devices on smithi099 2024-09-10T18:14:58.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:58 smithi133 ceph-mon[31214]: Adjusting osd_memory_target on smithi099 to 16064M 2024-09-10T18:14:58.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:58 smithi099 ceph-mon[27037]: Detected new or changed devices on smithi099 2024-09-10T18:14:58.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:58 smithi099 ceph-mon[27037]: Adjusting osd_memory_target on smithi099 to 16064M 2024-09-10T18:14:58.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:58 smithi099 ceph-mon[37557]: Detected new or changed devices on smithi099 2024-09-10T18:14:58.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:58 smithi099 ceph-mon[37557]: Adjusting osd_memory_target on smithi099 to 16064M 2024-09-10T18:14:59.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:14:59 smithi099 ceph-mon[27037]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:14:59.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:14:59 smithi099 ceph-mon[37557]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:14:59.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:14:59 smithi133 ceph-mon[31214]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:00 smithi099 ceph-mon[27037]: from='client.34142 v1:172.21.15.99:0/3241826883' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:15:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:00 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:15:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:00 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:15:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:00 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:00.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:00 smithi099 ceph-mon[37557]: from='client.34142 v1:172.21.15.99:0/3241826883' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:15:00.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:00 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:15:00.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:00 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:15:00.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:00 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:00.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:00 smithi133 ceph-mon[31214]: from='client.34142 v1:172.21.15.99:0/3241826883' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:15:00.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:00 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:15:00.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:00 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:15:00.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:00 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:01.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:01 smithi099 ceph-mon[27037]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:01.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:01 smithi099 ceph-mon[37557]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:01.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:01 smithi133 ceph-mon[31214]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:02.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:02 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/4050880314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d08b46a-4527-4e05-b267-fee4e9a52b38"}]: dispatch 2024-09-10T18:15:02.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:02 smithi133 ceph-mon[31214]: from='client.34148 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d08b46a-4527-4e05-b267-fee4e9a52b38"}]: dispatch 2024-09-10T18:15:02.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:02 smithi133 ceph-mon[31214]: from='client.34148 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d08b46a-4527-4e05-b267-fee4e9a52b38"}]': finished 2024-09-10T18:15:02.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:02 smithi133 ceph-mon[31214]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:15:02.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:02 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:02.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:02 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/891632296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:15:02.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:02 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/4050880314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d08b46a-4527-4e05-b267-fee4e9a52b38"}]: dispatch 2024-09-10T18:15:02.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:02 smithi099 ceph-mon[27037]: from='client.34148 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d08b46a-4527-4e05-b267-fee4e9a52b38"}]: dispatch 2024-09-10T18:15:02.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:02 smithi099 ceph-mon[27037]: from='client.34148 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d08b46a-4527-4e05-b267-fee4e9a52b38"}]': finished 2024-09-10T18:15:02.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:02 smithi099 ceph-mon[27037]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:15:02.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:02 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:02.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:02 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/891632296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:15:02.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:02 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/4050880314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d08b46a-4527-4e05-b267-fee4e9a52b38"}]: dispatch 2024-09-10T18:15:02.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:02 smithi099 ceph-mon[37557]: from='client.34148 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d08b46a-4527-4e05-b267-fee4e9a52b38"}]: dispatch 2024-09-10T18:15:02.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:02 smithi099 ceph-mon[37557]: from='client.34148 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d08b46a-4527-4e05-b267-fee4e9a52b38"}]': finished 2024-09-10T18:15:02.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:02 smithi099 ceph-mon[37557]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:15:02.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:02 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:02.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:02 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/891632296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:15:03.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:03 smithi133 ceph-mon[31214]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:03.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:03 smithi099 ceph-mon[27037]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:03.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:03 smithi099 ceph-mon[37557]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:05.476 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:05 smithi099 ceph-mon[27037]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:05.477 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:05 smithi099 ceph-mon[37557]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:05.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:05 smithi133 ceph-mon[31214]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:07.421 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:07 smithi099 ceph-mon[27037]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:07.421 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:07 smithi099 ceph-mon[37557]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:07.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:07 smithi133 ceph-mon[31214]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:08.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:08 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:15:08.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:08 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:08.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:08 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:15:08.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:08 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:08.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:08 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:15:08.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:08 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:09.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:09 smithi133 ceph-mon[31214]: Deploying daemon osd.1 on smithi099 2024-09-10T18:15:09.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:09 smithi133 ceph-mon[31214]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:09.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:09 smithi099 ceph-mon[27037]: Deploying daemon osd.1 on smithi099 2024-09-10T18:15:09.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:09 smithi099 ceph-mon[27037]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:09.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:09 smithi099 ceph-mon[37557]: Deploying daemon osd.1 on smithi099 2024-09-10T18:15:09.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:09 smithi099 ceph-mon[37557]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:11.392 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:11 smithi099 ceph-mon[27037]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:11.393 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:11 smithi099 ceph-mon[37557]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:11.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:11 smithi133 ceph-mon[31214]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:13.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:13 smithi099 ceph-mon[27037]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:13.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:13 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:15:13.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:13 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:13.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:13 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:13.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:13 smithi099 ceph-mon[37557]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:13.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:13 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:15:13.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:13 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:13.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:13 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:13.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:13 smithi133 ceph-mon[31214]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:13.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:13 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:15:13.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:13 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:13.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:13 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:15.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:15 smithi133 ceph-mon[31214]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:15.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:15 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:15.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:15 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:15.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:15 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:15.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:15 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:15:15.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:15 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:15.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:15 smithi099 ceph-mon[27037]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:15.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:15 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:15.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:15 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:15.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:15 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:15.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:15 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:15:15.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:15 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:15.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:15 smithi099 ceph-mon[37557]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:15.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:15 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:15.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:15 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:15.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:15 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:15.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:15 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:15:15.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:15 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:15.574 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 1 on host 'smithi099' 2024-09-10T18:15:16.159 DEBUG:teuthology.orchestra.run.smithi099:osd.1> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T18:15:16.161 INFO:tasks.cephadm:Deploying osd.2 on smithi099 with /dev/vg_nvme/lv_2... 2024-09-10T18:15:16.162 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:15:16.434 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:16 smithi099 ceph-mon[27037]: from='osd.1 v1:172.21.15.99:6805/719640887' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:15:16.434 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:16 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:15:16.434 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:16 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:16.434 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:16 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:16.435 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:16 smithi099 ceph-mon[37557]: from='osd.1 v1:172.21.15.99:6805/719640887' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:15:16.435 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:15:16.435 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:16.435 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:16 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:16.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:16 smithi133 ceph-mon[31214]: from='osd.1 v1:172.21.15.99:6805/719640887' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:15:16.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:16 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:15:16.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:16 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:16.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:16 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:17.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:17 smithi099 ceph-mon[27037]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:17.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:17 smithi099 ceph-mon[27037]: from='osd.1 v1:172.21.15.99:6805/719640887' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:15:17.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:17 smithi099 ceph-mon[27037]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:15:17.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:17 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:17.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:17 smithi099 ceph-mon[27037]: from='osd.1 v1:172.21.15.99:6805/719640887' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:15:17.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:17 smithi099 ceph-mon[37557]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:17.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:17 smithi099 ceph-mon[37557]: from='osd.1 v1:172.21.15.99:6805/719640887' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:15:17.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:17 smithi099 ceph-mon[37557]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:15:17.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:17 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:17.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:17 smithi099 ceph-mon[37557]: from='osd.1 v1:172.21.15.99:6805/719640887' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:15:17.256 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:15:17 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1[47455]: 2024-09-10T18:15:17.190+0000 7ff438577640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T18:15:17.256 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:15:17 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1[47455]: 2024-09-10T18:15:17.194+0000 7ff433baf640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:15:17.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:17 smithi133 ceph-mon[31214]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:17.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:17 smithi133 ceph-mon[31214]: from='osd.1 v1:172.21.15.99:6805/719640887' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:15:17.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:17 smithi133 ceph-mon[31214]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:15:17.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:17 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:17.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:17 smithi133 ceph-mon[31214]: from='osd.1 v1:172.21.15.99:6805/719640887' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:15:17.759 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:15:18.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:18 smithi133 ceph-mon[31214]: from='osd.1 v1:172.21.15.99:6805/719640887' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-10T18:15:18.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:18 smithi133 ceph-mon[31214]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:15:18.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:18 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:18.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:18 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:18.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:18 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:18.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:18 smithi099 ceph-mon[27037]: from='osd.1 v1:172.21.15.99:6805/719640887' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-10T18:15:18.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:18 smithi099 ceph-mon[27037]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:15:18.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:18 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:18.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:18 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:18.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:18 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:18.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:18 smithi099 ceph-mon[37557]: from='osd.1 v1:172.21.15.99:6805/719640887' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-10T18:15:18.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:18 smithi099 ceph-mon[37557]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:15:18.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:18 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:18.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:18 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:18.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:18 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:19.475 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: purged_snaps scrub starts 2024-09-10T18:15:19.476 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: purged_snaps scrub ok 2024-09-10T18:15:19.476 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:19.476 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: osd.1 v1:172.21.15.99:6805/719640887 boot 2024-09-10T18:15:19.476 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:15:19.476 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:19.476 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:19.476 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:19.476 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:19.476 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:19.476 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:19.476 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:19.477 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:15:19.477 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:19.477 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: purged_snaps scrub starts 2024-09-10T18:15:19.477 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: purged_snaps scrub ok 2024-09-10T18:15:19.477 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:19.477 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: osd.1 v1:172.21.15.99:6805/719640887 boot 2024-09-10T18:15:19.477 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:15:19.477 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:19.477 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:19.477 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:19.477 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:19.478 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:19.478 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:19.478 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:19.478 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:15:19.478 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:19 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:19.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: purged_snaps scrub starts 2024-09-10T18:15:19.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: purged_snaps scrub ok 2024-09-10T18:15:19.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:15:19.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: osd.1 v1:172.21.15.99:6805/719640887 boot 2024-09-10T18:15:19.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:15:19.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:15:19.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:19.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:19.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:19.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:19.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:19.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:19.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:15:19.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:19 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:19.939 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:15:20.027 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:15:20.041 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-09-10T18:15:20.041 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-09-10T18:15:20.041 INFO:teuthology.orchestra.run.smithi099.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124686 s, 841 MB/s 2024-09-10T18:15:20.041 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-09-10T18:15:20.347 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:20 smithi099 ceph-mon[27037]: Detected new or changed devices on smithi099 2024-09-10T18:15:20.347 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:20 smithi099 ceph-mon[27037]: Adjusting osd_memory_target on smithi099 to 8032M 2024-09-10T18:15:20.347 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:20 smithi099 ceph-mon[27037]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:15:20.347 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:20 smithi099 ceph-mon[37557]: Detected new or changed devices on smithi099 2024-09-10T18:15:20.347 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:20 smithi099 ceph-mon[37557]: Adjusting osd_memory_target on smithi099 to 8032M 2024-09-10T18:15:20.347 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:20 smithi099 ceph-mon[37557]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:15:20.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:20 smithi133 ceph-mon[31214]: Detected new or changed devices on smithi099 2024-09-10T18:15:20.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:20 smithi133 ceph-mon[31214]: Adjusting osd_memory_target on smithi099 to 8032M 2024-09-10T18:15:20.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:20 smithi133 ceph-mon[31214]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:15:20.655 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_2 2024-09-10T18:15:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:15:21.300 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:21 smithi099 ceph-mon[27037]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:21.301 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:21 smithi099 ceph-mon[37557]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:21.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:21 smithi133 ceph-mon[31214]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:23.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:23 smithi133 ceph-mon[31214]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:15:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:15:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:23 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:23 smithi099 ceph-mon[27037]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:15:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:15:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:23 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:23.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:23 smithi099 ceph-mon[37557]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:23.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:15:23.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:15:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:23 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:24.442 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:24 smithi099 ceph-mon[27037]: from='client.14268 v1:172.21.15.99:0/725137626' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:15:24.442 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:24 smithi099 ceph-mon[37557]: from='client.14268 v1:172.21.15.99:0/725137626' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:15:24.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:24 smithi133 ceph-mon[31214]: from='client.14268 v1:172.21.15.99:0/725137626' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:15:25.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:25 smithi099 ceph-mon[37557]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:25.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:25 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2682469572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "283d2b7c-557b-435a-a0b0-391352159d7a"}]: dispatch 2024-09-10T18:15:25.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:25 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2682469572' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "283d2b7c-557b-435a-a0b0-391352159d7a"}]': finished 2024-09-10T18:15:25.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:25 smithi099 ceph-mon[37557]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:15:25.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:25 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:25.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:25 smithi099 ceph-mon[27037]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:25.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:25 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2682469572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "283d2b7c-557b-435a-a0b0-391352159d7a"}]: dispatch 2024-09-10T18:15:25.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:25 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2682469572' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "283d2b7c-557b-435a-a0b0-391352159d7a"}]': finished 2024-09-10T18:15:25.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:25 smithi099 ceph-mon[27037]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:15:25.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:25 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:25.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:25 smithi133 ceph-mon[31214]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:25.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:25 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2682469572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "283d2b7c-557b-435a-a0b0-391352159d7a"}]: dispatch 2024-09-10T18:15:25.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:25 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2682469572' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "283d2b7c-557b-435a-a0b0-391352159d7a"}]': finished 2024-09-10T18:15:25.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:25 smithi133 ceph-mon[31214]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:15:25.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:25 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:26.495 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:26 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/902387931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:15:26.507 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:26 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/902387931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:15:26.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:26 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/902387931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:15:27.485 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:27 smithi099 ceph-mon[27037]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:27.485 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:27 smithi099 ceph-mon[37557]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:27.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:27 smithi133 ceph-mon[31214]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:29.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:29 smithi133 ceph-mon[31214]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:29.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:29 smithi099 ceph-mon[27037]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:29.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:29 smithi099 ceph-mon[37557]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:31.460 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:31 smithi099 ceph-mon[27037]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:31.460 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:31 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:15:31.460 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:31 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:31.460 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:31 smithi099 ceph-mon[37557]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:31.460 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:31 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:15:31.460 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:31 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:31.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:31 smithi133 ceph-mon[31214]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:31.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:31 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:15:31.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:31 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:32.414 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:32 smithi099 ceph-mon[27037]: Deploying daemon osd.2 on smithi099 2024-09-10T18:15:32.414 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:32 smithi099 ceph-mon[37557]: Deploying daemon osd.2 on smithi099 2024-09-10T18:15:32.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:32 smithi133 ceph-mon[31214]: Deploying daemon osd.2 on smithi099 2024-09-10T18:15:33.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:33 smithi099 ceph-mon[27037]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:33.489 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:33 smithi099 ceph-mon[37557]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:33.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:33 smithi133 ceph-mon[31214]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:35.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:35 smithi099 ceph-mon[27037]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:35.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:35 smithi099 ceph-mon[37557]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:35.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:35 smithi133 ceph-mon[31214]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:36.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:36 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:15:36.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:36 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:36.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:36 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:36.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:36 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:15:36.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:36 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:36.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:36 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:36.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:36 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:15:36.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:36 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:36.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:36 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:37.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:37 smithi099 ceph-mon[27037]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:37.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:37 smithi099 ceph-mon[37557]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:37.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:37 smithi133 ceph-mon[31214]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:38.785 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 2 on host 'smithi099' 2024-09-10T18:15:39.387 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:39 smithi099 ceph-mon[27037]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:39.388 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:39 smithi099 ceph-mon[27037]: from='osd.2 v1:172.21.15.99:6809/592364138' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:15:39.388 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:39 smithi099 ceph-mon[27037]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:15:39.388 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:39 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:39.388 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:39 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:39.388 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:39 smithi099 ceph-mon[37557]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:39.388 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:39 smithi099 ceph-mon[37557]: from='osd.2 v1:172.21.15.99:6809/592364138' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:15:39.388 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:39 smithi099 ceph-mon[37557]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:15:39.388 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:39 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:39.389 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:39 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:39.412 DEBUG:teuthology.orchestra.run.smithi099:osd.2> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T18:15:39.416 INFO:tasks.cephadm:Deploying osd.3 on smithi099 with /dev/vg_nvme/lv_1... 2024-09-10T18:15:39.416 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T18:15:39.710 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:15:39.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:39 smithi133 ceph-mon[31214]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:39.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:39 smithi133 ceph-mon[31214]: from='osd.2 v1:172.21.15.99:6809/592364138' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:15:39.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:39 smithi133 ceph-mon[31214]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:15:39.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:39 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:39.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:39 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:40.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[37557]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:15:40.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[37557]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:15:40.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:40.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[37557]: from='osd.2 v1:172.21.15.99:6809/592364138' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:15:40.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[37557]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:15:40.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:40.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:40.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[27037]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:15:40.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[27037]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:15:40.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:40.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[27037]: from='osd.2 v1:172.21.15.99:6809/592364138' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:15:40.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[27037]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:15:40.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:40.507 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:40.507 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2[53449]: 2024-09-10T18:15:40.256+0000 7effd94e4640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T18:15:40.507 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:15:40 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2[53449]: 2024-09-10T18:15:40.259+0000 7effd4b1c640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:15:40.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:40 smithi133 ceph-mon[31214]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:15:40.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:40 smithi133 ceph-mon[31214]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:15:40.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:40 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:40.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:40 smithi133 ceph-mon[31214]: from='osd.2 v1:172.21.15.99:6809/592364138' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:15:40.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:40 smithi133 ceph-mon[31214]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:15:40.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:40 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:40.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:40 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:41.380 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:41 smithi099 ceph-mon[27037]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:41.380 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:41 smithi099 ceph-mon[27037]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-10T18:15:41.380 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:41 smithi099 ceph-mon[27037]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:15:41.380 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:41 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:41.381 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:41 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:41.381 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:41 smithi099 ceph-mon[37557]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:41.381 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:41 smithi099 ceph-mon[37557]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-10T18:15:41.381 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:41 smithi099 ceph-mon[37557]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:15:41.381 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:41 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:41.381 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:41 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T18:15:41.490 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T18:15:41.506 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-09-10T18:15:41.506 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-09-10T18:15:41.506 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0143265 s, 732 MB/s 2024-09-10T18:15:41.506 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-09-10T18:15:41.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:41 smithi133 ceph-mon[31214]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:41.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:41 smithi133 ceph-mon[31214]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-10T18:15:41.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:41 smithi133 ceph-mon[31214]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:15:41.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:41 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:41.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:41 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:42.497 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_1 2024-09-10T18:15:42.521 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:42 smithi099 ceph-mon[27037]: purged_snaps scrub starts 2024-09-10T18:15:42.522 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:42 smithi099 ceph-mon[27037]: purged_snaps scrub ok 2024-09-10T18:15:42.522 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:42 smithi099 ceph-mon[27037]: osd.2 v1:172.21.15.99:6809/592364138 boot 2024-09-10T18:15:42.522 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:42 smithi099 ceph-mon[27037]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:15:42.522 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:42 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:42.522 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:42 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:15:42.522 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:42 smithi099 ceph-mon[37557]: purged_snaps scrub starts 2024-09-10T18:15:42.522 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:42 smithi099 ceph-mon[37557]: purged_snaps scrub ok 2024-09-10T18:15:42.522 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:42 smithi099 ceph-mon[37557]: osd.2 v1:172.21.15.99:6809/592364138 boot 2024-09-10T18:15:42.522 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:42 smithi099 ceph-mon[37557]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:15:42.523 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:42 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:42.523 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:42 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:15:42.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:42 smithi133 ceph-mon[31214]: purged_snaps scrub starts 2024-09-10T18:15:42.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:42 smithi133 ceph-mon[31214]: purged_snaps scrub ok 2024-09-10T18:15:42.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:42 smithi133 ceph-mon[31214]: osd.2 v1:172.21.15.99:6809/592364138 boot 2024-09-10T18:15:42.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:42 smithi133 ceph-mon[31214]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:15:42.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:42 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:15:42.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:42 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:15:43.103 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:15:43.422 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:43 smithi099 ceph-mon[27037]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:43.423 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:43 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:15:43.423 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:43 smithi099 ceph-mon[27037]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:15:43.423 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:43 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:15:43.423 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:43 smithi099 ceph-mon[37557]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:43.423 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:43 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:15:43.423 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:43 smithi099 ceph-mon[37557]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:15:43.423 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:43 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:15:43.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:43 smithi133 ceph-mon[31214]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:15:43.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:43 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:15:43.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:43 smithi133 ceph-mon[31214]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:15:43.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:43 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:15:43.755 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:15:43 smithi099 sudo[54960]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:15:43.755 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:15:43 smithi099 sudo[54960]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:15:43.755 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:15:43 smithi099 sudo[54960]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:15:43.755 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:15:43 smithi099 sudo[54960]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:15:44.101 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:15:43 smithi099 sudo[57228]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:15:44.101 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:15:43 smithi099 sudo[57228]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:15:44.101 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:15:43 smithi099 sudo[57228]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:15:44.101 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:15:43 smithi099 sudo[57228]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:15:44.363 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:44 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:15:44.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:44 smithi099 ceph-mon[27037]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:15:44.364 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:44 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:44.364 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:44 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:15:44.364 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:44 smithi099 ceph-mon[37557]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:15:44.364 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:44 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:44.364 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57502]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:15:44.364 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57502]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:15:44.365 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57502]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:15:44.365 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57502]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:15:44.619 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57508]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:15:44.620 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57508]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:15:44.620 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57508]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:15:44.620 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57508]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:15:44.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:44 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:15:44.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:44 smithi133 ceph-mon[31214]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:15:44.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:44 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:44.888 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57511]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:15:44.889 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57511]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:15:44.889 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57511]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:15:44.889 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57511]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:15:45.143 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57580]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:15:45.143 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57580]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:15:45.143 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57580]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:15:45.144 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:15:44 smithi099 sudo[57580]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:15:45.406 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 sudo[57605]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:15:45.407 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 sudo[57605]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:15:45.407 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 sudo[57605]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:15:45.407 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 sudo[57605]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:15:45.670 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:15:45.671 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:15:45.671 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:15:45.671 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.671 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.671 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:45.672 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:45.672 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:45.672 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.672 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:45.672 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:15:45.672 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.672 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:15:45.672 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:45.672 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:15:45.672 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.672 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:15:45.673 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:15:45.673 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:15:45.673 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 sudo[33745]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:15:45.673 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 sudo[33745]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:15:45.673 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 sudo[33745]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:15:45.673 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 sudo[33745]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:15:45.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 sudo[57613]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:15:45.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 sudo[57613]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:15:45.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 sudo[57613]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:15:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 sudo[57613]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:15:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:15:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:15:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:15:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:45.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:15:45.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:15:45.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:45.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:15:45.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:15:45.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:15:45.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:15:45.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:15:45.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:15:45.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:15:45.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:45.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:45.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:15:45.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:45.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:15:45.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:15:45.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:45.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:15:45.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:45.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:15:45.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:15:45.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:15:45.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 sudo[33748]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:15:45.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 sudo[33748]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:15:45.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 sudo[33748]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:15:45.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:45 smithi133 sudo[33748]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:15:46.366 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 sudo[57763]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:15:46.367 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 sudo[57763]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:15:46.367 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 sudo[57763]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:15:46.367 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 sudo[57763]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:15:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: Detected new or changed devices on smithi099 2024-09-10T18:15:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: Adjusting osd_memory_target on smithi099 to 5354M 2024-09-10T18:15:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:15:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:15:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:15:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:15:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:15:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:15:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:15:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:15:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:46.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:15:46.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:46.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:15:46.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:15:46.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:15:46.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: Detected new or changed devices on smithi099 2024-09-10T18:15:46.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: Adjusting osd_memory_target on smithi099 to 5354M 2024-09-10T18:15:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:15:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:15:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:15:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:15:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:15:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:15:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:15:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:15:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:15:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:15:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:15:46.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:15:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 sudo[57780]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:15:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 sudo[57780]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:15:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 sudo[57780]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:15:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 sudo[57780]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:15:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: Detected new or changed devices on smithi099 2024-09-10T18:15:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: Adjusting osd_memory_target on smithi099 to 5354M 2024-09-10T18:15:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:15:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:15:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:15:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:15:46.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:15:46.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:15:46.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:15:46.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:15:46.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:46.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:15:46.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:15:46.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:15:46.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:15:46.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:15:47.651 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:47 smithi099 ceph-mon[27037]: from='client.34202 v1:172.21.15.99:0/3918434569' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:15:47.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:47 smithi099 ceph-mon[27037]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:15:47.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:47 smithi099 ceph-mon[27037]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:15:47.652 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:47 smithi099 ceph-mon[27037]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:15:47.652 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:47 smithi099 ceph-mon[37557]: from='client.34202 v1:172.21.15.99:0/3918434569' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:15:47.652 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:47 smithi099 ceph-mon[37557]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:15:47.652 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:47 smithi099 ceph-mon[37557]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:15:47.652 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:47 smithi099 ceph-mon[37557]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:15:47.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:47 smithi133 ceph-mon[31214]: from='client.34202 v1:172.21.15.99:0/3918434569' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:15:47.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:47 smithi133 ceph-mon[31214]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:15:47.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:47 smithi133 ceph-mon[31214]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:15:47.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:47 smithi133 ceph-mon[31214]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:15:48.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:48 smithi133 ceph-mon[31214]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:15:48.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:48 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/357613869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "842ef8a8-5e78-4c71-83b2-dba621e4234b"}]: dispatch 2024-09-10T18:15:48.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:48 smithi133 ceph-mon[31214]: from='client.34211 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "842ef8a8-5e78-4c71-83b2-dba621e4234b"}]: dispatch 2024-09-10T18:15:48.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:48 smithi133 ceph-mon[31214]: from='client.34211 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "842ef8a8-5e78-4c71-83b2-dba621e4234b"}]': finished 2024-09-10T18:15:48.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:48 smithi133 ceph-mon[31214]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:15:48.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:48 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:15:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:48 smithi099 ceph-mon[27037]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:15:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:48 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/357613869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "842ef8a8-5e78-4c71-83b2-dba621e4234b"}]: dispatch 2024-09-10T18:15:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:48 smithi099 ceph-mon[27037]: from='client.34211 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "842ef8a8-5e78-4c71-83b2-dba621e4234b"}]: dispatch 2024-09-10T18:15:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:48 smithi099 ceph-mon[27037]: from='client.34211 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "842ef8a8-5e78-4c71-83b2-dba621e4234b"}]': finished 2024-09-10T18:15:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:48 smithi099 ceph-mon[27037]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:15:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:48 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:15:48.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:48 smithi099 ceph-mon[37557]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:15:48.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:48 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/357613869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "842ef8a8-5e78-4c71-83b2-dba621e4234b"}]: dispatch 2024-09-10T18:15:48.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:48 smithi099 ceph-mon[37557]: from='client.34211 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "842ef8a8-5e78-4c71-83b2-dba621e4234b"}]: dispatch 2024-09-10T18:15:48.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:48 smithi099 ceph-mon[37557]: from='client.34211 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "842ef8a8-5e78-4c71-83b2-dba621e4234b"}]': finished 2024-09-10T18:15:48.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:48 smithi099 ceph-mon[37557]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:15:48.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:48 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:15:49.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:49 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1585626696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:15:49.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:49 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1585626696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:15:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:49 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1585626696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:15:50.732 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:50 smithi099 ceph-mon[27037]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.3 KiB/s rd, 142 KiB/s wr, 10 op/s 2024-09-10T18:15:50.732 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:50 smithi099 ceph-mon[37557]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.3 KiB/s rd, 142 KiB/s wr, 10 op/s 2024-09-10T18:15:50.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:50 smithi133 ceph-mon[31214]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.3 KiB/s rd, 142 KiB/s wr, 10 op/s 2024-09-10T18:15:52.959 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:52 smithi099 ceph-mon[27037]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.4 KiB/s rd, 104 KiB/s wr, 7 op/s 2024-09-10T18:15:52.959 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:52 smithi099 ceph-mon[37557]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.4 KiB/s rd, 104 KiB/s wr, 7 op/s 2024-09-10T18:15:53.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:52 smithi133 ceph-mon[31214]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.4 KiB/s rd, 104 KiB/s wr, 7 op/s 2024-09-10T18:15:55.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:54 smithi133 ceph-mon[31214]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.9 KiB/s rd, 82 KiB/s wr, 5 op/s 2024-09-10T18:15:55.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:54 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:15:55.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:54 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:55.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:54 smithi133 ceph-mon[31214]: Deploying daemon osd.3 on smithi099 2024-09-10T18:15:55.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:54 smithi099 ceph-mon[27037]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.9 KiB/s rd, 82 KiB/s wr, 5 op/s 2024-09-10T18:15:55.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:54 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:15:55.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:54 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:55.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:54 smithi099 ceph-mon[27037]: Deploying daemon osd.3 on smithi099 2024-09-10T18:15:55.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:54 smithi099 ceph-mon[37557]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.9 KiB/s rd, 82 KiB/s wr, 5 op/s 2024-09-10T18:15:55.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:54 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:15:55.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:54 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:15:55.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:54 smithi099 ceph-mon[37557]: Deploying daemon osd.3 on smithi099 2024-09-10T18:15:57.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:56 smithi133 ceph-mon[31214]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:15:57.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:56 smithi099 ceph-mon[27037]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:15:57.436 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:56 smithi099 ceph-mon[37557]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:15:59.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:58 smithi099 ceph-mon[27037]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:15:59.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:58 smithi099 ceph-mon[37557]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:15:59.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:58 smithi133 ceph-mon[31214]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:16:00.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:59 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:00.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:59 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:00.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:15:59 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:00.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:59 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:00.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:59 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:00.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:15:59 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:00.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:59 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:00.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:59 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:00.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:15:59 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:01.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:00 smithi099 ceph-mon[27037]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 66 KiB/s wr, 4 op/s 2024-09-10T18:16:01.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:00 smithi099 ceph-mon[37557]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 66 KiB/s wr, 4 op/s 2024-09-10T18:16:01.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:00 smithi133 ceph-mon[31214]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 66 KiB/s wr, 4 op/s 2024-09-10T18:16:02.240 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 3 on host 'smithi099' 2024-09-10T18:16:02.877 DEBUG:teuthology.orchestra.run.smithi099:osd.3> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T18:16:02.919 INFO:tasks.cephadm:Deploying osd.4 on smithi133 with /dev/vg_nvme/lv_4... 2024-09-10T18:16:02.920 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:16:03.167 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:16:03.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:02 smithi133 ceph-mon[31214]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:16:03.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:02 smithi133 ceph-mon[31214]: from='osd.3 v1:172.21.15.99:6813/3452474565' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:16:03.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:02 smithi133 ceph-mon[31214]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:16:03.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:02 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:03.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:02 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:03.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:02 smithi099 ceph-mon[27037]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:16:03.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:02 smithi099 ceph-mon[27037]: from='osd.3 v1:172.21.15.99:6813/3452474565' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:16:03.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:02 smithi099 ceph-mon[27037]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:16:03.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:02 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:03.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:02 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:03.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:02 smithi099 ceph-mon[37557]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:16:03.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:02 smithi099 ceph-mon[37557]: from='osd.3 v1:172.21.15.99:6813/3452474565' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:16:03.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:02 smithi099 ceph-mon[37557]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:16:03.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:02 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:03.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:02 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:04.022 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:03 smithi133 ceph-mon[31214]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:16:04.022 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:03 smithi133 ceph-mon[31214]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:16:04.023 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:03 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:04.023 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:03 smithi133 ceph-mon[31214]: from='osd.3 v1:172.21.15.99:6813/3452474565' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:16:04.023 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:03 smithi133 ceph-mon[31214]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:16:04.023 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:03 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:04.023 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:03 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:04.192 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[27037]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:16:04.192 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[27037]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:16:04.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:04.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[27037]: from='osd.3 v1:172.21.15.99:6813/3452474565' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:16:04.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[27037]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:16:04.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:04.194 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:04.194 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[37557]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:16:04.194 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[37557]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:16:04.195 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:04.195 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[37557]: from='osd.3 v1:172.21.15.99:6813/3452474565' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:16:04.195 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[37557]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-10T18:16:04.195 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:04.196 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:04.196 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3[59505]: 2024-09-10T18:16:03.910+0000 7f319afe7640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T18:16:04.196 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:16:03 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3[59505]: 2024-09-10T18:16:03.914+0000 7f319461b640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:16:04.650 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:16:04.734 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:16:04.748 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-09-10T18:16:04.748 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-09-10T18:16:04.748 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114345 s, 917 MB/s 2024-09-10T18:16:04.748 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-09-10T18:16:05.175 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:04 smithi099 ceph-mon[27037]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:16:05.176 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:04 smithi099 ceph-mon[27037]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-10T18:16:05.176 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:04 smithi099 ceph-mon[27037]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:16:05.176 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:04 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:05.176 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:04 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:05.176 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:04 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:05.176 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:04 smithi099 ceph-mon[37557]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:16:05.176 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:04 smithi099 ceph-mon[37557]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-10T18:16:05.176 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:04 smithi099 ceph-mon[37557]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:16:05.176 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:04 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:05.176 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:04 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:05.177 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:04 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:05.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:04 smithi133 ceph-mon[31214]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:16:05.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:04 smithi133 ceph-mon[31214]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-10T18:16:05.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:04 smithi133 ceph-mon[31214]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:16:05.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:04 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:05.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:04 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:05.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:04 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:05.320 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_4 2024-09-10T18:16:05.575 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:16:06.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:05 smithi133 ceph-mon[31214]: purged_snaps scrub starts 2024-09-10T18:16:06.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:05 smithi133 ceph-mon[31214]: purged_snaps scrub ok 2024-09-10T18:16:06.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:05 smithi133 ceph-mon[31214]: osd.3 v1:172.21.15.99:6813/3452474565 boot 2024-09-10T18:16:06.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:05 smithi133 ceph-mon[31214]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:16:06.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:05 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:06.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:05 smithi099 ceph-mon[27037]: purged_snaps scrub starts 2024-09-10T18:16:06.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:05 smithi099 ceph-mon[27037]: purged_snaps scrub ok 2024-09-10T18:16:06.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:05 smithi099 ceph-mon[27037]: osd.3 v1:172.21.15.99:6813/3452474565 boot 2024-09-10T18:16:06.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:05 smithi099 ceph-mon[27037]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:16:06.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:05 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:06.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:05 smithi099 ceph-mon[37557]: purged_snaps scrub starts 2024-09-10T18:16:06.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:05 smithi099 ceph-mon[37557]: purged_snaps scrub ok 2024-09-10T18:16:06.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:05 smithi099 ceph-mon[37557]: osd.3 v1:172.21.15.99:6813/3452474565 boot 2024-09-10T18:16:06.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:05 smithi099 ceph-mon[37557]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:16:06.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:05 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:16:07.136 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:16:07.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:16:07.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:07.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:07.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:07.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:07.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:07.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:07.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.138 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:07.138 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:07.138 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:07.138 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:06 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:16:07.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:16:07.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:16:07.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:16:07.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:07.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:07.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:07.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:07.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:07.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:07.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:07.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:07.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:07.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:07.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:06 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:08.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:07 smithi133 ceph-mon[31214]: Detected new or changed devices on smithi099 2024-09-10T18:16:08.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:07 smithi133 ceph-mon[31214]: Adjusting osd_memory_target on smithi099 to 4016M 2024-09-10T18:16:08.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:07 smithi133 ceph-mon[31214]: from='client.24229 v1:172.21.15.133:0/1629579070' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:16:08.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:16:08.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:16:08.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:08.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:07 smithi099 ceph-mon[27037]: Detected new or changed devices on smithi099 2024-09-10T18:16:08.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:07 smithi099 ceph-mon[27037]: Adjusting osd_memory_target on smithi099 to 4016M 2024-09-10T18:16:08.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:07 smithi099 ceph-mon[27037]: from='client.24229 v1:172.21.15.133:0/1629579070' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:16:08.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:07 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:16:08.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:07 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:16:08.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:07 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:08.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:07 smithi099 ceph-mon[37557]: Detected new or changed devices on smithi099 2024-09-10T18:16:08.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:07 smithi099 ceph-mon[37557]: Adjusting osd_memory_target on smithi099 to 4016M 2024-09-10T18:16:08.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:07 smithi099 ceph-mon[37557]: from='client.24229 v1:172.21.15.133:0/1629579070' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:16:08.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:07 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:16:08.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:07 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:16:08.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:07 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:09.195 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:08 smithi133 ceph-mon[31214]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:09.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:08 smithi099 ceph-mon[27037]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:09.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:08 smithi099 ceph-mon[37557]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:10.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:09 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/1270727863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86a9c557-6eee-4f19-97f5-5397ae7edd50"}]: dispatch 2024-09-10T18:16:10.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:09 smithi133 ceph-mon[31214]: from='client.24235 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86a9c557-6eee-4f19-97f5-5397ae7edd50"}]: dispatch 2024-09-10T18:16:10.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:09 smithi133 ceph-mon[31214]: from='client.24235 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86a9c557-6eee-4f19-97f5-5397ae7edd50"}]': finished 2024-09-10T18:16:10.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:09 smithi133 ceph-mon[31214]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:16:10.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:09 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:10.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:09 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/1270727863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86a9c557-6eee-4f19-97f5-5397ae7edd50"}]: dispatch 2024-09-10T18:16:10.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:09 smithi099 ceph-mon[27037]: from='client.24235 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86a9c557-6eee-4f19-97f5-5397ae7edd50"}]: dispatch 2024-09-10T18:16:10.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:09 smithi099 ceph-mon[27037]: from='client.24235 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86a9c557-6eee-4f19-97f5-5397ae7edd50"}]': finished 2024-09-10T18:16:10.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:09 smithi099 ceph-mon[27037]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:16:10.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:09 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:10.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:09 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/1270727863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86a9c557-6eee-4f19-97f5-5397ae7edd50"}]: dispatch 2024-09-10T18:16:10.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:09 smithi099 ceph-mon[37557]: from='client.24235 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86a9c557-6eee-4f19-97f5-5397ae7edd50"}]: dispatch 2024-09-10T18:16:10.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:09 smithi099 ceph-mon[37557]: from='client.24235 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86a9c557-6eee-4f19-97f5-5397ae7edd50"}]': finished 2024-09-10T18:16:10.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:09 smithi099 ceph-mon[37557]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:16:10.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:09 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:11.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:10 smithi133 ceph-mon[31214]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:11.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:10 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/3919136476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:16:11.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:10 smithi099 ceph-mon[27037]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:11.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:10 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/3919136476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:16:11.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:10 smithi099 ceph-mon[37557]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:11.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:10 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/3919136476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:16:13.224 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:12 smithi133 ceph-mon[31214]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:13.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:12 smithi099 ceph-mon[27037]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:13.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:12 smithi099 ceph-mon[37557]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:15.213 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:14 smithi133 ceph-mon[31214]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:15.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:14 smithi099 ceph-mon[27037]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:15.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:14 smithi099 ceph-mon[37557]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:16.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:15 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:16:16.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:15 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:16.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:15 smithi133 ceph-mon[31214]: Deploying daemon osd.4 on smithi133 2024-09-10T18:16:16.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:15 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:16:16.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:15 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:16.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:15 smithi099 ceph-mon[27037]: Deploying daemon osd.4 on smithi133 2024-09-10T18:16:16.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:15 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:16:16.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:15 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:16.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:15 smithi099 ceph-mon[37557]: Deploying daemon osd.4 on smithi133 2024-09-10T18:16:17.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:16 smithi133 ceph-mon[31214]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:17.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:16 smithi099 ceph-mon[27037]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:17.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:16 smithi099 ceph-mon[37557]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:19.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:18 smithi133 ceph-mon[31214]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:19.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:18 smithi099 ceph-mon[27037]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:19.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:18 smithi099 ceph-mon[37557]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:19.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:19 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:19.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:19 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:19.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:19 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:20.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:19 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:20.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:19 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:20.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:19 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:20.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:19 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:20.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:19 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:20.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:19 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:21.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:20 smithi133 ceph-mon[31214]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:21.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:20 smithi099 ceph-mon[27037]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:21.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:20 smithi099 ceph-mon[37557]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:22.720 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 4 on host 'smithi133' 2024-09-10T18:16:22.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:22 smithi133 ceph-mon[31214]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:22.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:22 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:22.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:22 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:22.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:22 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:22.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:22 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:23.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:22 smithi099 ceph-mon[27037]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:23.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:22 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:23.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:22 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:23.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:22 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:23.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:22 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:23.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:22 smithi099 ceph-mon[37557]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:23.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:22 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:23.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:22 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:23.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:22 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:23.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:22 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:23.295 DEBUG:teuthology.orchestra.run.smithi133:osd.4> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T18:16:23.302 INFO:tasks.cephadm:Deploying osd.5 on smithi133 with /dev/vg_nvme/lv_3... 2024-09-10T18:16:23.302 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:16:24.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:23 smithi133 ceph-mon[31214]: from='osd.4 v1:172.21.15.133:6800/81144752' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:16:24.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:23 smithi133 ceph-mon[31214]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:16:24.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:23 smithi099 ceph-mon[27037]: from='osd.4 v1:172.21.15.133:6800/81144752' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:16:24.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:23 smithi099 ceph-mon[27037]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:16:24.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:23 smithi099 ceph-mon[37557]: from='osd.4 v1:172.21.15.133:6800/81144752' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:16:24.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:23 smithi099 ceph-mon[37557]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:16:24.284 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:16:25.241 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:16:24 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:16:24.982+0000 7fe09667c640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T18:16:25.241 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:16:24 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:16:24.986+0000 7fe08f4af640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:16:25.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:24 smithi133 ceph-mon[31214]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:25.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:24 smithi133 ceph-mon[31214]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:16:25.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:24 smithi133 ceph-mon[31214]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:16:25.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:24 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:25.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:24 smithi133 ceph-mon[31214]: from='osd.4 v1:172.21.15.133:6800/81144752' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:16:25.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:24 smithi133 ceph-mon[31214]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:16:25.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:24 smithi099 ceph-mon[27037]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:25.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:24 smithi099 ceph-mon[27037]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:16:25.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:24 smithi099 ceph-mon[27037]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:16:25.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:24 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:25.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:24 smithi099 ceph-mon[27037]: from='osd.4 v1:172.21.15.133:6800/81144752' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:16:25.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:24 smithi099 ceph-mon[27037]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:16:25.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:24 smithi099 ceph-mon[37557]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:25.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:24 smithi099 ceph-mon[37557]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:16:25.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:24 smithi099 ceph-mon[37557]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:16:25.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:24 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:25.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:24 smithi099 ceph-mon[37557]: from='osd.4 v1:172.21.15.133:6800/81144752' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:16:25.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:24 smithi099 ceph-mon[37557]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:16:25.985 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:16:26.097 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:16:26.113 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-09-10T18:16:26.114 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-09-10T18:16:26.114 INFO:teuthology.orchestra.run.smithi133.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0141833 s, 739 MB/s 2024-09-10T18:16:26.114 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 2024-09-10T18:16:26.114 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-09-10T18:16:26.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:25 smithi133 ceph-mon[31214]: purged_snaps scrub starts 2024-09-10T18:16:26.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:25 smithi133 ceph-mon[31214]: purged_snaps scrub ok 2024-09-10T18:16:26.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:25 smithi133 ceph-mon[31214]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T18:16:26.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:25 smithi133 ceph-mon[31214]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:16:26.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:25 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:26.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:25 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:26.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:25 smithi099 ceph-mon[27037]: purged_snaps scrub starts 2024-09-10T18:16:26.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:25 smithi099 ceph-mon[27037]: purged_snaps scrub ok 2024-09-10T18:16:26.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:25 smithi099 ceph-mon[27037]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T18:16:26.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:25 smithi099 ceph-mon[27037]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:16:26.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:25 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:26.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:25 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:26.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:25 smithi099 ceph-mon[37557]: purged_snaps scrub starts 2024-09-10T18:16:26.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:25 smithi099 ceph-mon[37557]: purged_snaps scrub ok 2024-09-10T18:16:26.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:25 smithi099 ceph-mon[37557]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T18:16:26.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:25 smithi099 ceph-mon[37557]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:16:26.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:25 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:26.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:25 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:26.786 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_3 2024-09-10T18:16:27.055 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:26 smithi133 ceph-mon[31214]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:27.055 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:26 smithi133 ceph-mon[31214]: osd.4 v1:172.21.15.133:6800/81144752 boot 2024-09-10T18:16:27.055 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:26 smithi133 ceph-mon[31214]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:16:27.055 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:26 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:27.055 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:26 smithi133 ceph-mon[31214]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:16:27.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:26 smithi099 ceph-mon[27037]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:27.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:26 smithi099 ceph-mon[27037]: osd.4 v1:172.21.15.133:6800/81144752 boot 2024-09-10T18:16:27.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:26 smithi099 ceph-mon[27037]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:16:27.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:26 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:27.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:26 smithi099 ceph-mon[27037]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:16:27.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:26 smithi099 ceph-mon[37557]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:16:27.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:26 smithi099 ceph-mon[37557]: osd.4 v1:172.21.15.133:6800/81144752 boot 2024-09-10T18:16:27.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:26 smithi099 ceph-mon[37557]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:16:27.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:26 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:16:27.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:26 smithi099 ceph-mon[37557]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:16:27.330 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:16:28.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: Detected new or changed devices on smithi133 2024-09-10T18:16:28.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:28.643 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: Adjusting osd_memory_target on smithi133 to 17088M 2024-09-10T18:16:28.643 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.643 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:28.643 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:28.643 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.643 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:28.643 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:28.643 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:28.643 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.643 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:28 smithi133 ceph-mon[31214]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:16:28.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: Detected new or changed devices on smithi133 2024-09-10T18:16:28.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:28.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: Adjusting osd_memory_target on smithi133 to 17088M 2024-09-10T18:16:28.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:28.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:28.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:28.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:28.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:28.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[27037]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:16:28.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: Detected new or changed devices on smithi133 2024-09-10T18:16:28.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:28.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: Adjusting osd_memory_target on smithi133 to 17088M 2024-09-10T18:16:28.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:28.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:28.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:28.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:28.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:28.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:28.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:28 smithi099 ceph-mon[37557]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:16:29.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:29 smithi133 ceph-mon[31214]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:29.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:16:29.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:16:29.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:29.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:29 smithi099 ceph-mon[27037]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:29.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:16:29.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:16:29.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:29.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:29 smithi099 ceph-mon[37557]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:29.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:16:29.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:16:29.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:30.554 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:30 smithi133 ceph-mon[31214]: from='client.24256 v1:172.21.15.133:0/158573168' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:16:30.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:30 smithi099 ceph-mon[27037]: from='client.24256 v1:172.21.15.133:0/158573168' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:16:30.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:30 smithi099 ceph-mon[37557]: from='client.24256 v1:172.21.15.133:0/158573168' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:16:31.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:31 smithi133 ceph-mon[31214]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-10T18:16:31.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:31 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/3382982622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f408970-5a8c-4957-88c3-532f00fe1461"}]: dispatch 2024-09-10T18:16:31.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:31 smithi133 ceph-mon[31214]: from='client.24262 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f408970-5a8c-4957-88c3-532f00fe1461"}]: dispatch 2024-09-10T18:16:31.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:31 smithi133 ceph-mon[31214]: from='client.24262 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f408970-5a8c-4957-88c3-532f00fe1461"}]': finished 2024-09-10T18:16:31.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:31 smithi133 ceph-mon[31214]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:16:31.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:31 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:31 smithi099 ceph-mon[27037]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-10T18:16:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:31 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/3382982622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f408970-5a8c-4957-88c3-532f00fe1461"}]: dispatch 2024-09-10T18:16:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:31 smithi099 ceph-mon[27037]: from='client.24262 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f408970-5a8c-4957-88c3-532f00fe1461"}]: dispatch 2024-09-10T18:16:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:31 smithi099 ceph-mon[27037]: from='client.24262 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f408970-5a8c-4957-88c3-532f00fe1461"}]': finished 2024-09-10T18:16:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:31 smithi099 ceph-mon[27037]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:16:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:31 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:31 smithi099 ceph-mon[37557]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-10T18:16:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:31 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/3382982622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f408970-5a8c-4957-88c3-532f00fe1461"}]: dispatch 2024-09-10T18:16:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:31 smithi099 ceph-mon[37557]: from='client.24262 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f408970-5a8c-4957-88c3-532f00fe1461"}]: dispatch 2024-09-10T18:16:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:31 smithi099 ceph-mon[37557]: from='client.24262 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f408970-5a8c-4957-88c3-532f00fe1461"}]': finished 2024-09-10T18:16:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:31 smithi099 ceph-mon[37557]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:16:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:31 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:32.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:32 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/2848239037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:16:32.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:32 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/2848239037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:16:32.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:32 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/2848239037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:16:33.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:33 smithi133 ceph-mon[31214]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-10T18:16:33.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:33 smithi099 ceph-mon[27037]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-10T18:16:33.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:33 smithi099 ceph-mon[37557]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-10T18:16:35.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:35 smithi133 ceph-mon[31214]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-10T18:16:35.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:35 smithi099 ceph-mon[27037]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-10T18:16:35.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:35 smithi099 ceph-mon[37557]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-10T18:16:37.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:37 smithi133 ceph-mon[31214]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:16:37.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:37 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:16:37.569 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:37 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:37.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:37 smithi099 ceph-mon[27037]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:16:37.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:37 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:16:37.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:37 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:37.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:37 smithi099 ceph-mon[37557]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:16:37.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:37 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:16:37.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:37 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:38.656 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:38 smithi133 ceph-mon[31214]: Deploying daemon osd.5 on smithi133 2024-09-10T18:16:38.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:38 smithi099 ceph-mon[27037]: Deploying daemon osd.5 on smithi133 2024-09-10T18:16:38.759 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:38 smithi099 ceph-mon[37557]: Deploying daemon osd.5 on smithi133 2024-09-10T18:16:39.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:39 smithi133 ceph-mon[31214]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T18:16:39.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:39 smithi099 ceph-mon[27037]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T18:16:39.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:39 smithi099 ceph-mon[37557]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T18:16:41.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:41 smithi133 ceph-mon[31214]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:41.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:41 smithi099 ceph-mon[27037]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:41.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:41 smithi099 ceph-mon[37557]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:42.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:42 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:42.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:42 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:42.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:42 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:42.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:42 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:42.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:42 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:42.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:42 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:42.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:42 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:42.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:42 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:42.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:42 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:43.655 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:43 smithi133 ceph-mon[31214]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:43.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:43 smithi099 ceph-mon[27037]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:43.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:43 smithi099 ceph-mon[37557]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:44.525 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 5 on host 'smithi133' 2024-09-10T18:16:44.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:44 smithi133 ceph-mon[31214]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:44.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:44 smithi099 ceph-mon[27037]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:44.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:44 smithi099 ceph-mon[37557]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:45.143 DEBUG:teuthology.orchestra.run.smithi133:osd.5> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T18:16:45.145 INFO:tasks.cephadm:Deploying osd.6 on smithi133 with /dev/vg_nvme/lv_2... 2024-09-10T18:16:45.145 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:16:45.438 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:16:45.788 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:45.788 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:45.788 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:45 smithi133 ceph-mon[31214]: from='osd.5 v1:172.21.15.133:6804/2255028820' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:16:45.788 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:45 smithi133 ceph-mon[31214]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:16:45.789 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:45.789 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:45 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:46.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:46.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:46.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:45 smithi099 ceph-mon[27037]: from='osd.5 v1:172.21.15.133:6804/2255028820' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:16:46.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:45 smithi099 ceph-mon[27037]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:16:46.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:46.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:45 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:46.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:46.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:46.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:45 smithi099 ceph-mon[37557]: from='osd.5 v1:172.21.15.133:6804/2255028820' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:16:46.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:45 smithi099 ceph-mon[37557]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:16:46.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:46.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:45 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:46.704 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:46 smithi133 ceph-mon[31214]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:16:46.704 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:46 smithi133 ceph-mon[31214]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:16:46.704 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:46.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:46 smithi133 ceph-mon[31214]: from='osd.5 v1:172.21.15.133:6804/2255028820' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:16:46.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:46 smithi133 ceph-mon[31214]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:16:46.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:46 smithi133 ceph-mon[31214]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:46.705 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:16:46 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:16:46.534+0000 7fca317d8640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T18:16:46.705 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:16:46 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:16:46.539+0000 7fca2be0e640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:16:47.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:46 smithi099 ceph-mon[27037]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:16:47.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:46 smithi099 ceph-mon[27037]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:16:47.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:47.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:46 smithi099 ceph-mon[27037]: from='osd.5 v1:172.21.15.133:6804/2255028820' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:16:47.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:46 smithi099 ceph-mon[27037]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:16:47.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:46 smithi099 ceph-mon[27037]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:47.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:46 smithi099 ceph-mon[37557]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:16:47.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:46 smithi099 ceph-mon[37557]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:16:47.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:47.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:46 smithi099 ceph-mon[37557]: from='osd.5 v1:172.21.15.133:6804/2255028820' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:16:47.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:46 smithi099 ceph-mon[37557]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:16:47.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:46 smithi099 ceph-mon[37557]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:16:47.849 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:47 smithi133 ceph-mon[31214]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T18:16:47.849 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:47 smithi133 ceph-mon[31214]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:16:47.850 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:47 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:47.850 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:47 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:47.850 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:47 smithi133 ceph-mon[31214]: osd.5 v1:172.21.15.133:6804/2255028820 boot 2024-09-10T18:16:47.850 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:47 smithi133 ceph-mon[31214]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T18:16:47.850 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:47 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:47.983 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:16:48.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[27037]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T18:16:48.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[27037]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:16:48.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:48.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:48.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[27037]: osd.5 v1:172.21.15.133:6804/2255028820 boot 2024-09-10T18:16:48.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[27037]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T18:16:48.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:48.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[37557]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T18:16:48.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[37557]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:16:48.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:48.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:48.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[37557]: osd.5 v1:172.21.15.133:6804/2255028820 boot 2024-09-10T18:16:48.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[37557]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T18:16:48.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:47 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:16:48.090 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:16:48.104 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-09-10T18:16:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-09-10T18:16:48.105 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128949 s, 813 MB/s 2024-09-10T18:16:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-09-10T18:16:48.932 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:48 smithi133 ceph-mon[31214]: purged_snaps scrub starts 2024-09-10T18:16:48.932 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:48 smithi133 ceph-mon[31214]: purged_snaps scrub ok 2024-09-10T18:16:48.932 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:48 smithi133 ceph-mon[31214]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:48.932 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:48 smithi133 ceph-mon[31214]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:16:48.955 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_2 2024-09-10T18:16:49.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:48 smithi099 ceph-mon[27037]: purged_snaps scrub starts 2024-09-10T18:16:49.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:48 smithi099 ceph-mon[27037]: purged_snaps scrub ok 2024-09-10T18:16:49.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:48 smithi099 ceph-mon[27037]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:49.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:48 smithi099 ceph-mon[27037]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:16:49.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:48 smithi099 ceph-mon[37557]: purged_snaps scrub starts 2024-09-10T18:16:49.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:48 smithi099 ceph-mon[37557]: purged_snaps scrub ok 2024-09-10T18:16:49.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:48 smithi099 ceph-mon[37557]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:49.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:48 smithi099 ceph-mon[37557]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:16:49.195 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:16:50.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:16:50.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:50.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: Detected new or changed devices on smithi133 2024-09-10T18:16:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: Adjusting osd_memory_target on smithi133 to 8544M 2024-09-10T18:16:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:50.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.760 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:16:50.760 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:50.760 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: Detected new or changed devices on smithi133 2024-09-10T18:16:50.760 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.760 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.760 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:50.761 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:50.761 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: Adjusting osd_memory_target on smithi133 to 8544M 2024-09-10T18:16:50.761 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.761 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:50.761 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:50.761 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.761 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:50.762 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:50.762 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:50.762 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:50 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.809 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:16:50.809 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:50.809 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: Detected new or changed devices on smithi133 2024-09-10T18:16:50.809 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.810 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.810 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:50.810 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:16:50.810 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: Adjusting osd_memory_target on smithi133 to 8544M 2024-09-10T18:16:50.810 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.810 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:50.810 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:50.810 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:50.810 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:16:50.810 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:50.810 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:16:50.811 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:50 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:16:51.889 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:51 smithi133 ceph-mon[31214]: from='client.24283 v1:172.21.15.133:0/433798413' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:16:51.890 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:51 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:16:51.890 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:51 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:16:51.890 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:51 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:52.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:51 smithi099 ceph-mon[27037]: from='client.24283 v1:172.21.15.133:0/433798413' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:16:52.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:51 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:16:52.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:51 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:16:52.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:51 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:52.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:51 smithi099 ceph-mon[37557]: from='client.24283 v1:172.21.15.133:0/433798413' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:16:52.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:51 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:16:52.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:51 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:16:52.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:51 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:16:52.820 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:52 smithi133 ceph-mon[31214]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:53.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:52 smithi099 ceph-mon[27037]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:53.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:52 smithi099 ceph-mon[37557]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:53.845 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:53 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/2111429961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92985eff-4bb3-408e-bc3f-6bcd6f97d50e"}]: dispatch 2024-09-10T18:16:53.845 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:53 smithi133 ceph-mon[31214]: from='client.24289 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92985eff-4bb3-408e-bc3f-6bcd6f97d50e"}]: dispatch 2024-09-10T18:16:53.845 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:53 smithi133 ceph-mon[31214]: from='client.24289 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92985eff-4bb3-408e-bc3f-6bcd6f97d50e"}]': finished 2024-09-10T18:16:53.845 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:53 smithi133 ceph-mon[31214]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T18:16:53.845 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:53 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:16:54.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:53 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/2111429961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92985eff-4bb3-408e-bc3f-6bcd6f97d50e"}]: dispatch 2024-09-10T18:16:54.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:53 smithi099 ceph-mon[27037]: from='client.24289 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92985eff-4bb3-408e-bc3f-6bcd6f97d50e"}]: dispatch 2024-09-10T18:16:54.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:53 smithi099 ceph-mon[27037]: from='client.24289 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92985eff-4bb3-408e-bc3f-6bcd6f97d50e"}]': finished 2024-09-10T18:16:54.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:53 smithi099 ceph-mon[27037]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T18:16:54.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:53 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:16:54.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:53 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/2111429961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92985eff-4bb3-408e-bc3f-6bcd6f97d50e"}]: dispatch 2024-09-10T18:16:54.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:53 smithi099 ceph-mon[37557]: from='client.24289 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92985eff-4bb3-408e-bc3f-6bcd6f97d50e"}]: dispatch 2024-09-10T18:16:54.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:53 smithi099 ceph-mon[37557]: from='client.24289 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92985eff-4bb3-408e-bc3f-6bcd6f97d50e"}]': finished 2024-09-10T18:16:54.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:53 smithi099 ceph-mon[37557]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T18:16:54.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:53 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:16:54.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:54 smithi133 ceph-mon[31214]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:54.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:54 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/95698423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:16:55.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:54 smithi099 ceph-mon[27037]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:55.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:54 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/95698423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:16:55.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:54 smithi099 ceph-mon[37557]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:55.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:54 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/95698423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:16:57.216 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:56 smithi133 ceph-mon[31214]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:57.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:56 smithi099 ceph-mon[27037]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:57.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:56 smithi099 ceph-mon[37557]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:59.138 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:58 smithi133 ceph-mon[31214]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:59.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:58 smithi099 ceph-mon[27037]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:16:59.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:58 smithi099 ceph-mon[37557]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:00.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:59 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:17:00.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:59 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:00.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:16:59 smithi133 ceph-mon[31214]: Deploying daemon osd.6 on smithi133 2024-09-10T18:17:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:59 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:17:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:59 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:16:59 smithi099 ceph-mon[27037]: Deploying daemon osd.6 on smithi133 2024-09-10T18:17:00.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:59 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:17:00.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:59 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:00.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:16:59 smithi099 ceph-mon[37557]: Deploying daemon osd.6 on smithi133 2024-09-10T18:17:01.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:00 smithi133 ceph-mon[31214]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:01.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:00 smithi099 ceph-mon[27037]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:01.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:00 smithi099 ceph-mon[37557]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:03.224 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:02 smithi133 ceph-mon[31214]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:03.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:02 smithi099 ceph-mon[27037]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:03.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:02 smithi099 ceph-mon[37557]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:04.181 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:03 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:04.182 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:03 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:04.182 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:03 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:04.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:03 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:04.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:03 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:04.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:03 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:04.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:03 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:04.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:03 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:04.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:03 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:05.167 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:04 smithi133 ceph-mon[31214]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:05.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:04 smithi099 ceph-mon[27037]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:05.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:04 smithi099 ceph-mon[37557]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:06.260 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 6 on host 'smithi133' 2024-09-10T18:17:07.234 DEBUG:teuthology.orchestra.run.smithi133:osd.6> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T18:17:07.236 INFO:tasks.cephadm:Deploying osd.7 on smithi133 with /dev/vg_nvme/lv_1... 2024-09-10T18:17:07.236 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T18:17:07.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:07 smithi133 ceph-mon[31214]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:07.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:07.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:07 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:07.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:07 smithi133 ceph-mon[31214]: from='osd.6 v1:172.21.15.133:6808/2733605656' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:17:07.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:07 smithi133 ceph-mon[31214]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:17:07.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:07 smithi099 ceph-mon[27037]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:07.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:07 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:07.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:07 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:07.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:07 smithi099 ceph-mon[27037]: from='osd.6 v1:172.21.15.133:6808/2733605656' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:17:07.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:07 smithi099 ceph-mon[27037]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:17:07.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:07 smithi099 ceph-mon[37557]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:07.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:07 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:07.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:07 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:07.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:07 smithi099 ceph-mon[37557]: from='osd.6 v1:172.21.15.133:6808/2733605656' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:17:07.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:07 smithi099 ceph-mon[37557]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:17:07.527 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:17:08.480 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:08 smithi133 ceph-mon[31214]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:17:08.480 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:08 smithi133 ceph-mon[31214]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:17:08.480 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:08 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:17:08.480 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:08 smithi133 ceph-mon[31214]: from='osd.6 v1:172.21.15.133:6808/2733605656' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:17:08.481 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:08 smithi133 ceph-mon[31214]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:17:08.481 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:08 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:08.481 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:08 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:08.481 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:17:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:17:08.267+0000 7f48dfc89640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T18:17:08.481 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:17:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:17:08.273+0000 7f48db2c1640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:17:08.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[27037]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:17:08.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[27037]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:17:08.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:17:08.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[27037]: from='osd.6 v1:172.21.15.133:6808/2733605656' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:17:08.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[27037]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:17:08.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:08.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:08.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[37557]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:17:08.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[37557]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:17:08.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:17:08.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[37557]: from='osd.6 v1:172.21.15.133:6808/2733605656' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:17:08.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[37557]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:17:08.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:08.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:08 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T18:17:09.232 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T18:17:09.246 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-09-10T18:17:09.247 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-09-10T18:17:09.247 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129983 s, 807 MB/s 2024-09-10T18:17:09.247 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-09-10T18:17:09.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:09 smithi133 ceph-mon[31214]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:09.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:09 smithi133 ceph-mon[31214]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T18:17:09.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:09 smithi133 ceph-mon[31214]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:17:09.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:09 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:17:09.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:09 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:17:09.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:09 smithi133 ceph-mon[31214]: osd.6 v1:172.21.15.133:6808/2733605656 boot 2024-09-10T18:17:09.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:09 smithi133 ceph-mon[31214]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T18:17:09.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:09 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:17:09.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[27037]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:09.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[27037]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T18:17:09.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[27037]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:17:09.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:17:09.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:17:09.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[27037]: osd.6 v1:172.21.15.133:6808/2733605656 boot 2024-09-10T18:17:09.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[27037]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T18:17:09.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:17:09.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[37557]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:17:09.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[37557]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T18:17:09.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[37557]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:17:09.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:17:09.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:17:09.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[37557]: osd.6 v1:172.21.15.133:6808/2733605656 boot 2024-09-10T18:17:09.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[37557]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T18:17:09.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:09 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:17:09.911 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_1 2024-09-10T18:17:10.392 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:17:10.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:10 smithi133 ceph-mon[31214]: purged_snaps scrub starts 2024-09-10T18:17:10.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:10 smithi133 ceph-mon[31214]: purged_snaps scrub ok 2024-09-10T18:17:10.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:10 smithi133 ceph-mon[31214]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:17:10.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:10 smithi133 ceph-mon[31214]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:11.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:10 smithi099 ceph-mon[27037]: purged_snaps scrub starts 2024-09-10T18:17:11.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:10 smithi099 ceph-mon[27037]: purged_snaps scrub ok 2024-09-10T18:17:11.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:10 smithi099 ceph-mon[27037]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:17:11.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:10 smithi099 ceph-mon[27037]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:11.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:10 smithi099 ceph-mon[37557]: purged_snaps scrub starts 2024-09-10T18:17:11.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:10 smithi099 ceph-mon[37557]: purged_snaps scrub ok 2024-09-10T18:17:11.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:10 smithi099 ceph-mon[37557]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:17:11.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:10 smithi099 ceph-mon[37557]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:11.877 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:11 smithi133 ceph-mon[31214]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:17:12.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:11 smithi099 ceph-mon[27037]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:17:12.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:11 smithi099 ceph-mon[37557]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:17:12.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:12 smithi133 ceph-mon[31214]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:12.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:12 smithi133 ceph-mon[31214]: from='client.24310 v1:172.21.15.133:0/2522788067' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:12.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:17:12.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:17:12.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:12 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:13.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:12 smithi099 ceph-mon[27037]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:13.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:12 smithi099 ceph-mon[27037]: from='client.24310 v1:172.21.15.133:0/2522788067' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:13.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:17:13.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:17:13.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:12 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:13.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:12 smithi099 ceph-mon[37557]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:13.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:12 smithi099 ceph-mon[37557]: from='client.24310 v1:172.21.15.133:0/2522788067' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:13.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:12 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:17:13.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:12 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:17:13.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:12 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:15.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:15.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: Detected new or changed devices on smithi133 2024-09-10T18:17:15.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:15.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:15.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:15.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: Adjusting osd_memory_target on smithi133 to 5696M 2024-09-10T18:17:15.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:15.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:15.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:15.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:15.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:15.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:14 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:15.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: Detected new or changed devices on smithi133 2024-09-10T18:17:15.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:15.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:15.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:15.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: Adjusting osd_memory_target on smithi133 to 5696M 2024-09-10T18:17:15.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:15.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:15.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:15.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:15.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:15.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:15.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: Detected new or changed devices on smithi133 2024-09-10T18:17:15.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:15.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:15.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:15.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: Adjusting osd_memory_target on smithi133 to 5696M 2024-09-10T18:17:15.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:15.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:15.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:15.258 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:15.259 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:15.259 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:15.259 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:14 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:16.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:15 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/3247185711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94ce876c-555c-47a1-9c35-22d82bf77e0c"}]: dispatch 2024-09-10T18:17:16.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:15 smithi133 ceph-mon[31214]: from='client.24316 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94ce876c-555c-47a1-9c35-22d82bf77e0c"}]: dispatch 2024-09-10T18:17:16.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:15 smithi133 ceph-mon[31214]: from='client.24316 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94ce876c-555c-47a1-9c35-22d82bf77e0c"}]': finished 2024-09-10T18:17:16.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:15 smithi133 ceph-mon[31214]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T18:17:16.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:15 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:16.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:15 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/4187175195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:17:16.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:15 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/3247185711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94ce876c-555c-47a1-9c35-22d82bf77e0c"}]: dispatch 2024-09-10T18:17:16.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:15 smithi099 ceph-mon[27037]: from='client.24316 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94ce876c-555c-47a1-9c35-22d82bf77e0c"}]: dispatch 2024-09-10T18:17:16.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:15 smithi099 ceph-mon[27037]: from='client.24316 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94ce876c-555c-47a1-9c35-22d82bf77e0c"}]': finished 2024-09-10T18:17:16.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:15 smithi099 ceph-mon[27037]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T18:17:16.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:15 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:16.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:15 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/4187175195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:17:16.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:15 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/3247185711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94ce876c-555c-47a1-9c35-22d82bf77e0c"}]: dispatch 2024-09-10T18:17:16.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:15 smithi099 ceph-mon[37557]: from='client.24316 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94ce876c-555c-47a1-9c35-22d82bf77e0c"}]: dispatch 2024-09-10T18:17:16.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:15 smithi099 ceph-mon[37557]: from='client.24316 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94ce876c-555c-47a1-9c35-22d82bf77e0c"}]': finished 2024-09-10T18:17:16.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:15 smithi099 ceph-mon[37557]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T18:17:16.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:15 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:16.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:15 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/4187175195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:17:17.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:17 smithi133 ceph-mon[31214]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:17.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:17 smithi099 ceph-mon[27037]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:17.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:17 smithi099 ceph-mon[37557]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:19.639 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:19 smithi133 ceph-mon[31214]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:19.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:19 smithi099 ceph-mon[27037]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:19.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:19 smithi099 ceph-mon[37557]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:21.558 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:21 smithi133 ceph-mon[31214]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:21.558 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:17:21.558 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:21 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:21.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:21 smithi099 ceph-mon[27037]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:21.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:17:21.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:21 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:21.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:21 smithi099 ceph-mon[37557]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:21.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:17:21.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:21 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:22.560 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:22 smithi133 ceph-mon[31214]: Deploying daemon osd.7 on smithi133 2024-09-10T18:17:22.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:22 smithi099 ceph-mon[27037]: Deploying daemon osd.7 on smithi133 2024-09-10T18:17:22.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:22 smithi099 ceph-mon[37557]: Deploying daemon osd.7 on smithi133 2024-09-10T18:17:23.563 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:23 smithi133 ceph-mon[31214]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:23.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:23 smithi099 ceph-mon[27037]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:23.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:23 smithi099 ceph-mon[37557]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:25.547 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:25 smithi133 ceph-mon[31214]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:25.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:25 smithi099 ceph-mon[27037]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:25.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:25 smithi099 ceph-mon[37557]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:26.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:26 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:26.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:26 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:26.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:26 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:26.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:26 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:26.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:26 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:26.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:26 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:26.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:26 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:26.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:26 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:26.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:26 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:27.417 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:27 smithi133 ceph-mon[31214]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:27.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:27 smithi099 ceph-mon[27037]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:27.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:27 smithi099 ceph-mon[37557]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:28.837 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 7 on host 'smithi133' 2024-09-10T18:17:29.404 DEBUG:teuthology.orchestra.run.smithi133:osd.7> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T18:17:29.406 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-10T18:17:29.407 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T18:17:29.427 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:29 smithi133 ceph-mon[31214]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:29.428 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:29.428 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:29 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:29.428 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:29 smithi133 ceph-mon[31214]: from='osd.7 v1:172.21.15.133:6812/3452975685' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:17:29.428 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:29 smithi133 ceph-mon[31214]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:17:29.665 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:17:29.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:29 smithi099 ceph-mon[27037]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:29.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:29.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:29 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:29.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:29 smithi099 ceph-mon[27037]: from='osd.7 v1:172.21.15.133:6812/3452975685' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:17:29.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:29 smithi099 ceph-mon[27037]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:17:29.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:29 smithi099 ceph-mon[37557]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:29.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:29.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:29 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:29.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:29 smithi099 ceph-mon[37557]: from='osd.7 v1:172.21.15.133:6812/3452975685' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:17:29.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:29 smithi099 ceph-mon[37557]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:17:30.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:30.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:30.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[27037]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:30.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[27037]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:17:30.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[27037]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:17:30.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:30.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[27037]: from='osd.7 v1:172.21.15.133:6812/3452975685' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:17:30.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[27037]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:17:30.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:30.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:30.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[37557]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:30.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[37557]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:17:30.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[37557]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:17:30.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:30.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[37557]: from='osd.7 v1:172.21.15.133:6812/3452975685' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:17:30.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:30 smithi099 ceph-mon[37557]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:17:30.837 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:30 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:30.837 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:30 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:30.838 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:30 smithi133 ceph-mon[31214]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:30.838 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:30 smithi133 ceph-mon[31214]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:17:30.838 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:30 smithi133 ceph-mon[31214]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:17:30.838 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:30 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:30.838 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:30 smithi133 ceph-mon[31214]: from='osd.7 v1:172.21.15.133:6812/3452975685' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:17:30.838 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:30 smithi133 ceph-mon[31214]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T18:17:31.098 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:17:30 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:17:30.845+0000 7fc455cf4640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T18:17:31.098 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:17:30 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:17:30.851+0000 7fc44eb27640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:17:31.449 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:17:32.069 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":47,"num_osds":8,"num_up_osds":7,"osd_up_since":1725992228,"num_in_osds":8,"osd_in_since":1725992235,"num_remapped_pgs":0} 2024-09-10T18:17:32.135 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:31 smithi133 ceph-mon[31214]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T18:17:32.135 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:31 smithi133 ceph-mon[31214]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:17:32.135 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:31 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:32.135 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:31 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:32.135 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:31 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2755723926' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:17:32.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:31 smithi099 ceph-mon[27037]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T18:17:32.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:31 smithi099 ceph-mon[27037]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:17:32.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:31 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:32.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:31 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:32.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:31 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2755723926' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:17:32.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:31 smithi099 ceph-mon[37557]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T18:17:32.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:31 smithi099 ceph-mon[37557]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:17:32.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:31 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:32.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:31 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:32.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:31 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2755723926' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:17:33.070 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T18:17:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: purged_snaps scrub starts 2024-09-10T18:17:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: purged_snaps scrub ok 2024-09-10T18:17:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: pgmap v156: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:17:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: osd.7 v1:172.21.15.133:6812/3452975685 boot 2024-09-10T18:17:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T18:17:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: Detected new or changed devices on smithi133 2024-09-10T18:17:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:33.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:33.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:33.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:33.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: Adjusting osd_memory_target on smithi133 to 4272M 2024-09-10T18:17:33.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:33.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:33.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:33.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:33.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:33.243 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:32 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: purged_snaps scrub starts 2024-09-10T18:17:33.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: purged_snaps scrub ok 2024-09-10T18:17:33.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: pgmap v156: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:33.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:17:33.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: osd.7 v1:172.21.15.133:6812/3452975685 boot 2024-09-10T18:17:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T18:17:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: Detected new or changed devices on smithi133 2024-09-10T18:17:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: Adjusting osd_memory_target on smithi133 to 4272M 2024-09-10T18:17:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:33.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:33.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:33.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:33.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:33.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: purged_snaps scrub starts 2024-09-10T18:17:33.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: purged_snaps scrub ok 2024-09-10T18:17:33.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: pgmap v156: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:17:33.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:17:33.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: osd.7 v1:172.21.15.133:6812/3452975685 boot 2024-09-10T18:17:33.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T18:17:33.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:17:33.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: Detected new or changed devices on smithi133 2024-09-10T18:17:33.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:33.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:33.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:33.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:17:33.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: Adjusting osd_memory_target on smithi133 to 4272M 2024-09-10T18:17:33.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.258 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:33.259 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:33.259 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.259 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:33.259 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:33.259 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:33.259 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:32 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:33.318 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:17:34.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:33 smithi099 ceph-mon[27037]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:17:34.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:33 smithi099 ceph-mon[27037]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:17:34.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:33 smithi099 ceph-mon[37557]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:17:34.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:33 smithi099 ceph-mon[37557]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:17:34.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:33 smithi133 ceph-mon[31214]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:17:34.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:33 smithi133 ceph-mon[31214]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:17:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:34 smithi099 ceph-mon[27037]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:17:35.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:34 smithi099 ceph-mon[37557]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:17:35.049 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:17:35.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:34 smithi133 ceph-mon[31214]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:17:35.752 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":50,"num_osds":8,"num_up_osds":8,"osd_up_since":1725992251,"num_in_osds":8,"osd_in_since":1725992235,"num_remapped_pgs":0} 2024-09-10T18:17:35.752 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:17:35.997 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:17:36.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:35 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3210270561' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:17:36.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:35 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3210270561' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:17:36.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:35 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3210270561' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:17:37.135 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:36 smithi099 ceph-mon[27037]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:17:37.135 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:36 smithi099 ceph-mon[37557]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:17:37.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:36 smithi133 ceph-mon[31214]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:17:37.798 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:17:37.799 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":50,"fsid":"e9807d90-6f9f-11ef-bcea-c7b262605968","created":"2024-09-10T18:12:43.787016+0000","modified":"2024-09-10T18:17:33.844351+0000","last_up_change":"2024-09-10T18:17:31.838123+0000","last_in_change":"2024-09-10T18:17:15.270408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:15:42.054570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"2d91a1a4-657a-4417-a71c-ac9576241c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6801","nonce":24904954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6802","nonce":24904954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6804","nonce":24904954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6803","nonce":24904954}]},"public_addr":"172.21.15.99:6801/24904954","cluster_addr":"172.21.15.99:6802/24904954","heartbeat_back_addr":"172.21.15.99:6804/24904954","heartbeat_front_addr":"172.21.15.99:6803/24904954","state":["exists","up"]},{"osd":1,"uuid":"8d08b46a-4527-4e05-b267-fee4e9a52b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6805","nonce":719640887}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6806","nonce":719640887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6808","nonce":719640887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6807","nonce":719640887}]},"public_addr":"172.21.15.99:6805/719640887","cluster_addr":"172.21.15.99:6806/719640887","heartbeat_back_addr":"172.21.15.99:6808/719640887","heartbeat_front_addr":"172.21.15.99:6807/719640887","state":["exists","up"]},{"osd":2,"uuid":"283d2b7c-557b-435a-a0b0-391352159d7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6809","nonce":592364138}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6810","nonce":592364138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6812","nonce":592364138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6811","nonce":592364138}]},"public_addr":"172.21.15.99:6809/592364138","cluster_addr":"172.21.15.99:6810/592364138","heartbeat_back_addr":"172.21.15.99:6812/592364138","heartbeat_front_addr":"172.21.15.99:6811/592364138","state":["exists","up"]},{"osd":3,"uuid":"842ef8a8-5e78-4c71-83b2-dba621e4234b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6813","nonce":3452474565}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6814","nonce":3452474565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6816","nonce":3452474565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6815","nonce":3452474565}]},"public_addr":"172.21.15.99:6813/3452474565","cluster_addr":"172.21.15.99:6814/3452474565","heartbeat_back_addr":"172.21.15.99:6816/3452474565","heartbeat_front_addr":"172.21.15.99:6815/3452474565","state":["exists","up"]},{"osd":4,"uuid":"86a9c557-6eee-4f19-97f5-5397ae7edd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6800","nonce":81144752}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6801","nonce":81144752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6803","nonce":81144752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6802","nonce":81144752}]},"public_addr":"172.21.15.133:6800/81144752","cluster_addr":"172.21.15.133:6801/81144752","heartbeat_back_addr":"172.21.15.133:6803/81144752","heartbeat_front_addr":"172.21.15.133:6802/81144752","state":["exists","up"]},{"osd":5,"uuid":"0f408970-5a8c-4957-88c3-532f00fe1461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6804","nonce":2255028820}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6805","nonce":2255028820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6807","nonce":2255028820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6806","nonce":2255028820}]},"public_addr":"172.21.15.133:6804/2255028820","cluster_addr":"172.21.15.133:6805/2255028820","heartbeat_back_addr":"172.21.15.133:6807/2255028820","heartbeat_front_addr":"172.21.15.133:6806/2255028820","state":["exists","up"]},{"osd":6,"uuid":"92985eff-4bb3-408e-bc3f-6bcd6f97d50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6808","nonce":2733605656}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6809","nonce":2733605656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6811","nonce":2733605656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6810","nonce":2733605656}]},"public_addr":"172.21.15.133:6808/2733605656","cluster_addr":"172.21.15.133:6809/2733605656","heartbeat_back_addr":"172.21.15.133:6811/2733605656","heartbeat_front_addr":"172.21.15.133:6810/2733605656","state":["exists","up"]},{"osd":7,"uuid":"94ce876c-555c-47a1-9c35-22d82bf77e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6812","nonce":3452975685}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6813","nonce":3452975685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6815","nonce":3452975685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6814","nonce":3452975685}]},"public_addr":"172.21.15.133:6812/3452975685","cluster_addr":"172.21.15.133:6813/3452975685","heartbeat_back_addr":"172.21.15.133:6815/3452975685","heartbeat_front_addr":"172.21.15.133:6814/3452975685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:14:53.163875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:15:16.351745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:15:39.625973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:16:03.018305+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:16:23.952366+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:16:45.610214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:17:07.929453+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:17:29.891109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/1048941956":"2024-09-11T18:13:33.759212+0000","172.21.15.99:0/193334044":"2024-09-11T18:13:33.759212+0000","172.21.15.99:0/463482381":"2024-09-11T18:13:33.759212+0000","172.21.15.99:0/1512987781":"2024-09-11T18:13:11.071109+0000","172.21.15.99:0/1960046719":"2024-09-11T18:13:11.071109+0000","172.21.15.99:6800/1682160141":"2024-09-11T18:13:33.759212+0000","172.21.15.99:6800/2446219887":"2024-09-11T18:13:11.071109+0000","172.21.15.99:0/215619363":"2024-09-11T18:13:11.071109+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:17:38.118 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:37 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2336907912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:17:38.118 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:37 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2336907912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:17:38.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:37 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2336907912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:17:38.544 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T18:15:42.054570+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-10T18:17:38.545 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T18:17:38.784 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:17:39.138 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:38 smithi099 ceph-mon[27037]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:17:39.138 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:38 smithi099 ceph-mon[27037]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:17:39.138 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:38 smithi099 ceph-mon[27037]: Cluster is now healthy 2024-09-10T18:17:39.139 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:38 smithi099 ceph-mon[37557]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:17:39.139 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:38 smithi099 ceph-mon[37557]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:17:39.139 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:38 smithi099 ceph-mon[37557]: Cluster is now healthy 2024-09-10T18:17:39.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:38 smithi133 ceph-mon[31214]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:17:39.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:38 smithi133 ceph-mon[31214]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:17:39.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:38 smithi133 ceph-mon[31214]: Cluster is now healthy 2024-09-10T18:17:40.639 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-09-10T18:17:40.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:40 smithi099 ceph-mon[27037]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:17:40.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:40 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1356883260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:17:40.990 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:40 smithi099 ceph-mon[37557]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:17:40.990 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:40 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1356883260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:17:41.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:40 smithi133 ceph-mon[31214]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:17:41.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:40 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1356883260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:17:41.410 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi099 2024-09-10T18:17:41.411 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi099=foo.a' 2024-09-10T18:17:41.666 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:17:43.017 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:42 smithi133 ceph-mon[31214]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T18:17:43.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:42 smithi099 ceph-mon[27037]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T18:17:43.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:42 smithi099 ceph-mon[37557]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T18:17:43.566 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled rgw.foo.a update... 2024-09-10T18:17:44.173 DEBUG:teuthology.orchestra.run.smithi099:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-10T18:17:44.175 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi133 2024-09-10T18:17:44.176 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-10T18:17:44.420 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:17:44.793 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:44 smithi133 ceph-mon[31214]: from='client.24358 v1:172.21.15.133:0/1456458973' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi099=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:44.793 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:44 smithi133 ceph-mon[31214]: Saving service rgw.foo.a spec with placement smithi099=foo.a;count:1 2024-09-10T18:17:44.794 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:44 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:44.794 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:44 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:44.794 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:44 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:44.794 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:44 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:44.794 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:44 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:44.794 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:44 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:17:44.794 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:44 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:17:44.794 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:44 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:44.794 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:44 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:44.794 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:44 smithi133 ceph-mon[31214]: Deploying daemon rgw.foo.a on smithi099 2024-09-10T18:17:44.794 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:44 smithi133 ceph-mon[31214]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:17:45.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[37557]: from='client.24358 v1:172.21.15.133:0/1456458973' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi099=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:45.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[37557]: Saving service rgw.foo.a spec with placement smithi099=foo.a;count:1 2024-09-10T18:17:45.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:45.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:45.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:45.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:45.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:45.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:17:45.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:17:45.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:45.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:45.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[37557]: Deploying daemon rgw.foo.a on smithi099 2024-09-10T18:17:45.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[37557]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:17:45.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[27037]: from='client.24358 v1:172.21.15.133:0/1456458973' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi099=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:45.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[27037]: Saving service rgw.foo.a spec with placement smithi099=foo.a;count:1 2024-09-10T18:17:45.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:45.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:45.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:45.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:45.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:45.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:17:45.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:17:45.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:45.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:45.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[27037]: Deploying daemon rgw.foo.a on smithi099 2024-09-10T18:17:45.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:44 smithi099 ceph-mon[27037]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:17:45.755 INFO:journalctl@ceph.rgw.foo.a.smithi099.stdout:Sep 10 18:17:45 smithi099 podman[64616]: 2024-09-10 18:17:45.681912401 +0000 UTC m=+0.145721615 container create 28bdc0b2c33d9d9fd01ce7927cdb9c3780355beeb380b24852fee49630d8fa7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-rgw-foo-a, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T18:17:46.160 INFO:journalctl@ceph.rgw.foo.a.smithi099.stdout:Sep 10 18:17:46 smithi099 podman[64616]: 2024-09-10 18:17:46.05949888 +0000 UTC m=+0.523308090 container init 28bdc0b2c33d9d9fd01ce7927cdb9c3780355beeb380b24852fee49630d8fa7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-rgw-foo-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T18:17:46.161 INFO:journalctl@ceph.rgw.foo.a.smithi099.stdout:Sep 10 18:17:46 smithi099 podman[64616]: 2024-09-10 18:17:46.06247496 +0000 UTC m=+0.526284171 container start 28bdc0b2c33d9d9fd01ce7927cdb9c3780355beeb380b24852fee49630d8fa7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T18:17:46.505 INFO:journalctl@ceph.rgw.foo.a.smithi099.stdout:Sep 10 18:17:46 smithi099 bash[64616]: 28bdc0b2c33d9d9fd01ce7927cdb9c3780355beeb380b24852fee49630d8fa7f 2024-09-10T18:17:46.505 INFO:journalctl@ceph.rgw.foo.a.smithi099.stdout:Sep 10 18:17:46 smithi099 systemd[1]: Started Ceph rgw.foo.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:17:46.902 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'datapool' created 2024-09-10T18:17:47.173 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[37557]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T18:17:47.174 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/1909862665' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:17:47.174 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[37557]: from='client.24364 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:17:47.174 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.174 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.174 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.174 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.174 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.174 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:47.174 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[27037]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T18:17:47.174 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/1909862665' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:17:47.174 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[27037]: from='client.24364 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:17:47.175 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.175 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.175 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.175 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.175 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.175 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:46 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:47.236 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:46 smithi133 ceph-mon[31214]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T18:17:47.236 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:46 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/1909862665' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:17:47.236 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:46 smithi133 ceph-mon[31214]: from='client.24364 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:17:47.236 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.236 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.236 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.237 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.237 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:47.237 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:46 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:17:47.513 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-10T18:17:47.758 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:17:48.190 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:47 smithi133 ceph-mon[31214]: Saving service rgw.foo.a spec with placement smithi099=foo.a;count:1 2024-09-10T18:17:48.190 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:47 smithi133 ceph-mon[31214]: from='client.24364 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:17:48.190 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:47 smithi133 ceph-mon[31214]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:17:48.190 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:47 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1310164021' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:17:48.190 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:47 smithi133 ceph-mon[31214]: from='client.34376 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:17:48.190 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:47 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:48.191 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:47 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:48.191 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:47 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:48.191 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:47 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[27037]: Saving service rgw.foo.a spec with placement smithi099=foo.a;count:1 2024-09-10T18:17:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[27037]: from='client.24364 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:17:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[27037]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:17:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1310164021' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:17:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[27037]: from='client.34376 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:17:48.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:48.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:48.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:48.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[37557]: Saving service rgw.foo.a spec with placement smithi099=foo.a;count:1 2024-09-10T18:17:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[37557]: from='client.24364 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:17:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[37557]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:17:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1310164021' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:17:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[37557]: from='client.34376 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:17:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:48.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:48.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:47 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:17:49.214 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:48 smithi133 ceph-mon[31214]: pgmap v168: 36 pgs: 7 creating+peering, 28 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:17:49.214 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:48 smithi133 ceph-mon[31214]: Checking dashboard <-> RGW credentials 2024-09-10T18:17:49.214 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:48 smithi133 ceph-mon[31214]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:17:49.214 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:48 smithi133 ceph-mon[31214]: from='client.34376 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:17:49.214 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:48 smithi133 ceph-mon[31214]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:17:49.215 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:48 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:48 smithi099 ceph-mon[27037]: pgmap v168: 36 pgs: 7 creating+peering, 28 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:17:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:48 smithi099 ceph-mon[27037]: Checking dashboard <-> RGW credentials 2024-09-10T18:17:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:48 smithi099 ceph-mon[27037]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:17:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:48 smithi099 ceph-mon[27037]: from='client.34376 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:17:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:48 smithi099 ceph-mon[27037]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:17:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:48 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:49.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:48 smithi099 ceph-mon[37557]: pgmap v168: 36 pgs: 7 creating+peering, 28 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:17:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:48 smithi099 ceph-mon[37557]: Checking dashboard <-> RGW credentials 2024-09-10T18:17:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:48 smithi099 ceph-mon[37557]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:17:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:48 smithi099 ceph-mon[37557]: from='client.34376 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:17:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:48 smithi099 ceph-mon[37557]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:17:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:48 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:50.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:49 smithi133 ceph-mon[31214]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:17:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:49 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1310164021' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:17:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:49 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2702605606' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:17:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:49 smithi133 ceph-mon[31214]: from='client.34376 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:17:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:49 smithi133 ceph-mon[31214]: from='client.34400 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:17:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:49 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/1002810853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:17:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:49 smithi133 ceph-mon[31214]: from='client.24403 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:17:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[27037]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:17:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1310164021' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:17:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2702605606' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:17:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[27037]: from='client.34376 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:17:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[27037]: from='client.34400 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:17:50.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/1002810853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:17:50.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[27037]: from='client.24403 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:17:50.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[37557]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:17:50.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1310164021' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:17:50.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2702605606' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:17:50.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[37557]: from='client.34376 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:17:50.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[37557]: from='client.34400 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:17:50.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/1002810853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:17:50.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:49 smithi099 ceph-mon[37557]: from='client.24403 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:17:51.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:50 smithi133 ceph-mon[31214]: pgmap v171: 68 pgs: 15 creating+peering, 47 unknown, 6 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 170 B/s wr, 0 op/s 2024-09-10T18:17:51.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:50 smithi133 ceph-mon[31214]: from='client.34376 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:17:51.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:50 smithi133 ceph-mon[31214]: from='client.34400 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:17:51.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:50 smithi133 ceph-mon[31214]: from='client.24403 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:17:51.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:50 smithi133 ceph-mon[31214]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:17:51.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:50 smithi099 ceph-mon[27037]: pgmap v171: 68 pgs: 15 creating+peering, 47 unknown, 6 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 170 B/s wr, 0 op/s 2024-09-10T18:17:51.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:50 smithi099 ceph-mon[27037]: from='client.34376 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:17:51.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:50 smithi099 ceph-mon[27037]: from='client.34400 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:17:51.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:50 smithi099 ceph-mon[27037]: from='client.24403 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:17:51.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:50 smithi099 ceph-mon[27037]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:17:51.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:50 smithi099 ceph-mon[37557]: pgmap v171: 68 pgs: 15 creating+peering, 47 unknown, 6 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 170 B/s wr, 0 op/s 2024-09-10T18:17:51.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:50 smithi099 ceph-mon[37557]: from='client.34376 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:17:51.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:50 smithi099 ceph-mon[37557]: from='client.34400 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:17:51.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:50 smithi099 ceph-mon[37557]: from='client.24403 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:17:51.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:50 smithi099 ceph-mon[37557]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:17:52.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:51 smithi133 ceph-mon[31214]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:17:52.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:51 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1310164021' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:17:52.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:51 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2702605606' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:17:52.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:51 smithi133 ceph-mon[31214]: from='client.34376 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:17:52.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:51 smithi133 ceph-mon[31214]: from='client.34400 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:17:52.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:51 smithi099 ceph-mon[27037]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:17:52.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:51 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1310164021' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:17:52.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:51 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2702605606' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:17:52.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:51 smithi099 ceph-mon[27037]: from='client.34376 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:17:52.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:51 smithi099 ceph-mon[27037]: from='client.34400 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:17:52.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:51 smithi099 ceph-mon[37557]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:17:52.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:51 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1310164021' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:17:52.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:51 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2702605606' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:17:52.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:51 smithi099 ceph-mon[37557]: from='client.34376 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:17:52.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:51 smithi099 ceph-mon[37557]: from='client.34400 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:17:52.529 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.133 --placement '1;smithi133=iscsi.a' 2024-09-10T18:17:52.775 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:17:53.187 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:52 smithi133 ceph-mon[31214]: pgmap v174: 100 pgs: 22 creating+peering, 35 unknown, 43 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:17:53.187 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:52 smithi133 ceph-mon[31214]: from='client.34376 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:17:53.187 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:52 smithi133 ceph-mon[31214]: from='client.34400 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:17:53.187 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:52 smithi133 ceph-mon[31214]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:17:53.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:52 smithi099 ceph-mon[27037]: pgmap v174: 100 pgs: 22 creating+peering, 35 unknown, 43 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:17:53.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:52 smithi099 ceph-mon[27037]: from='client.34376 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:17:53.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:52 smithi099 ceph-mon[27037]: from='client.34400 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:17:53.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:52 smithi099 ceph-mon[27037]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:17:53.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:52 smithi099 ceph-mon[37557]: pgmap v174: 100 pgs: 22 creating+peering, 35 unknown, 43 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:17:53.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:52 smithi099 ceph-mon[37557]: from='client.34376 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:17:53.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:52 smithi099 ceph-mon[37557]: from='client.34400 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:17:53.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:52 smithi099 ceph-mon[37557]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:17:54.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:53 smithi133 ceph-mon[31214]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:17:54.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:53 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/571023785' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:17:54.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:53 smithi133 ceph-mon[31214]: from='client.34415 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:17:54.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:53 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/734400679' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:17:54.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:53 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:17:54.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:53 smithi099 ceph-mon[27037]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:17:54.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:53 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/571023785' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:17:54.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:53 smithi099 ceph-mon[27037]: from='client.34415 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:17:54.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:53 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/734400679' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:17:54.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:53 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:17:54.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:53 smithi099 ceph-mon[37557]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:17:54.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:53 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/571023785' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:17:54.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:53 smithi099 ceph-mon[37557]: from='client.34415 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:17:54.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:53 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/734400679' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:17:54.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:53 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:17:54.592 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled iscsi.datapool update... 2024-09-10T18:17:55.164 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:54 smithi133 ceph-mon[31214]: pgmap v177: 132 pgs: 19 creating+peering, 43 unknown, 70 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T18:17:55.165 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:54 smithi133 ceph-mon[31214]: from='client.34415 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:17:55.165 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:54 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/734400679' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:17:55.165 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:54 smithi133 ceph-mon[31214]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:17:55.165 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:54 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/571023785' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:17:55.165 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:54 smithi133 ceph-mon[31214]: from='client.34415 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:17:55.166 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:54 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/734400679' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:17:55.166 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:54 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:55.193 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-10T18:17:55.194 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:17:55.194 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:17:55.220 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[37557]: pgmap v177: 132 pgs: 19 creating+peering, 43 unknown, 70 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T18:17:55.220 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[37557]: from='client.34415 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:17:55.220 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/734400679' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:17:55.220 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[37557]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:17:55.220 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/571023785' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:17:55.220 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[37557]: from='client.34415 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:17:55.220 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/734400679' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:17:55.220 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:55.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[27037]: pgmap v177: 132 pgs: 19 creating+peering, 43 unknown, 70 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T18:17:55.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[27037]: from='client.34415 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:17:55.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/734400679' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:17:55.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[27037]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:17:55.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/571023785' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:17:55.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[27037]: from='client.34415 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:17:55.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/734400679' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:17:55.221 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:54 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:55.224 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T18:17:55.225 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:17:55.254 DEBUG:teuthology.orchestra.run.smithi133:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-10T18:17:55.296 INFO:tasks.cephadm:Adding prometheus.a on smithi133 2024-09-10T18:17:55.296 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi133=a' 2024-09-10T18:17:56.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:55 smithi133 ceph-mon[31214]: from='client.24421 v1:172.21.15.133:0/3876433240' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.133", "placement": "1;smithi133=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:56.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:55 smithi133 ceph-mon[31214]: Saving service iscsi.datapool spec with placement smithi133=iscsi.a;count:1 2024-09-10T18:17:56.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:55 smithi133 ceph-mon[31214]: from='client.34415 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:17:56.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:55 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/734400679' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:17:56.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:55 smithi133 ceph-mon[31214]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:17:56.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:55 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:56.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:55 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:56.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:55 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:56.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:55 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:17:56.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:55 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:17:56.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:55 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:56.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:55 smithi133 ceph-mon[31214]: Deploying daemon iscsi.iscsi.a on smithi133 2024-09-10T18:17:56.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[27037]: from='client.24421 v1:172.21.15.133:0/3876433240' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.133", "placement": "1;smithi133=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:56.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[27037]: Saving service iscsi.datapool spec with placement smithi133=iscsi.a;count:1 2024-09-10T18:17:56.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[27037]: from='client.34415 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:17:56.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/734400679' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:17:56.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[27037]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:17:56.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:56.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:56.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:56.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:17:56.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:17:56.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:56.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[27037]: Deploying daemon iscsi.iscsi.a on smithi133 2024-09-10T18:17:56.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[37557]: from='client.24421 v1:172.21.15.133:0/3876433240' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.133", "placement": "1;smithi133=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:17:56.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[37557]: Saving service iscsi.datapool spec with placement smithi133=iscsi.a;count:1 2024-09-10T18:17:56.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[37557]: from='client.34415 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:17:56.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/734400679' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:17:56.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[37557]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:17:56.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:56.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:56.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:56.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:17:56.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:17:56.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:17:56.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:55 smithi099 ceph-mon[37557]: Deploying daemon iscsi.iscsi.a on smithi133 2024-09-10T18:17:56.622 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:17:57.066 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:56 smithi133 ceph-mon[31214]: pgmap v180: 132 pgs: 11 creating+peering, 33 unknown, 88 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T18:17:57.066 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:56 smithi133 ceph-mon[31214]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:17:57.067 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:56 smithi133 ceph-mon[31214]: Cluster is now healthy 2024-09-10T18:17:57.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:56 smithi099 ceph-mon[27037]: pgmap v180: 132 pgs: 11 creating+peering, 33 unknown, 88 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T18:17:57.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:56 smithi099 ceph-mon[27037]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:17:57.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:56 smithi099 ceph-mon[27037]: Cluster is now healthy 2024-09-10T18:17:57.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:56 smithi099 ceph-mon[37557]: pgmap v180: 132 pgs: 11 creating+peering, 33 unknown, 88 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T18:17:57.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:56 smithi099 ceph-mon[37557]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:17:57.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:56 smithi099 ceph-mon[37557]: Cluster is now healthy 2024-09-10T18:17:58.758 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled prometheus update... 2024-09-10T18:17:59.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:58 smithi133 ceph-mon[31214]: pgmap v181: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s rd, 4.7 KiB/s wr, 135 op/s 2024-09-10T18:17:59.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:58 smithi133 ceph-mon[31214]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:17:59.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:58 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:59.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:58 smithi099 ceph-mon[27037]: pgmap v181: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s rd, 4.7 KiB/s wr, 135 op/s 2024-09-10T18:17:59.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:58 smithi099 ceph-mon[27037]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:17:59.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:58 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:59.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:58 smithi099 ceph-mon[37557]: pgmap v181: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s rd, 4.7 KiB/s wr, 135 op/s 2024-09-10T18:17:59.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:58 smithi099 ceph-mon[37557]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:17:59.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:58 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:17:59.447 DEBUG:teuthology.orchestra.run.smithi133:prometheus.a> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-10T18:17:59.451 INFO:tasks.cephadm:Adding node-exporter.a on smithi099 2024-09-10T18:17:59.451 INFO:tasks.cephadm:Adding node-exporter.b on smithi133 2024-09-10T18:17:59.451 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi099=a;smithi133=b' 2024-09-10T18:17:59.673 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:17:59 smithi133 podman[58518]: 2024-09-10 18:17:59.601344802 +0000 UTC m=+1.182492949 container start 64ebfc7716f38f63ec30bbbe956ce9d2147bfa86b05b3b18867ff501714178f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T18:17:59.751 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:17:59.949 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:17:59 smithi133 bash[58518]: 64ebfc7716f38f63ec30bbbe956ce9d2147bfa86b05b3b18867ff501714178f3 2024-09-10T18:17:59.949 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:17:59 smithi133 systemd[1]: Started Ceph iscsi.iscsi.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:17:59.949 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:17:59 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug Started the configuration object watcher 2024-09-10T18:17:59.949 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:17:59 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug Checking for config object changes every 1s 2024-09-10T18:17:59.949 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:17:59 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug Processing osd blocklist entries for this node 2024-09-10T18:18:00.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:59 smithi133 ceph-mon[31214]: from='client.24442 v1:172.21.15.133:0/3299748353' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi133=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:00.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:59 smithi133 ceph-mon[31214]: Saving service prometheus spec with placement smithi133=a;count:1 2024-09-10T18:18:00.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:59 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:00.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:59 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:00.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:59 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:00.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:59 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:00.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:17:59 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:00.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug Reading the configuration object to update local LIO configuration 2024-09-10T18:18:00.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug Configuration does not have an entry for this host(smithi133.front.sepia.ceph.com) - nothing to define to LIO 2024-09-10T18:18:00.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-10T18:18:00.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: * Environment: production 2024-09-10T18:18:00.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:18:00.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: Use a production WSGI server instead. 2024-09-10T18:18:00.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: * Debug mode: off 2024-09-10T18:18:00.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug * Running on all addresses. 2024-09-10T18:18:00.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:18:00.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: * Running on all addresses. 2024-09-10T18:18:00.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:18:00.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug * Running on http://[fe80::ec4:7aff:fe88:8087]:5000/ (Press CTRL+C to quit) 2024-09-10T18:18:00.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: * Running on http://[fe80::ec4:7aff:fe88:8087]:5000/ (Press CTRL+C to quit) 2024-09-10T18:18:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[27037]: from='client.24442 v1:172.21.15.133:0/3299748353' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi133=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[27037]: Saving service prometheus spec with placement smithi133=a;count:1 2024-09-10T18:18:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:00.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:00.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[37557]: from='client.24442 v1:172.21.15.133:0/3299748353' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi133=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:00.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[37557]: Saving service prometheus spec with placement smithi133=a;count:1 2024-09-10T18:18:00.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:00.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:00.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:00.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:00.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:17:59 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:01.152 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-mon[31214]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:18:01.152 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-mon[31214]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 77 KiB/s rd, 6.0 KiB/s wr, 187 op/s 2024-09-10T18:18:01.152 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/3596397954' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:18:01.152 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:00 smithi133 ceph-mon[31214]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:18:01.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:00 smithi099 ceph-mon[27037]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:18:01.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:00 smithi099 ceph-mon[27037]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 77 KiB/s rd, 6.0 KiB/s wr, 187 op/s 2024-09-10T18:18:01.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:00 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/3596397954' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:18:01.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:00 smithi099 ceph-mon[27037]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:18:01.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:00 smithi099 ceph-mon[37557]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:18:01.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:00 smithi099 ceph-mon[37557]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 77 KiB/s rd, 6.0 KiB/s wr, 187 op/s 2024-09-10T18:18:01.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:00 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/3596397954' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:18:01.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:00 smithi099 ceph-mon[37557]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:18:01.538 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled node-exporter update... 2024-09-10T18:18:02.212 DEBUG:teuthology.orchestra.run.smithi099:node-exporter.a> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-10T18:18:02.215 DEBUG:teuthology.orchestra.run.smithi133:node-exporter.b> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-10T18:18:02.218 INFO:tasks.cephadm:Adding alertmanager.a on smithi099 2024-09-10T18:18:02.218 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi099=a' 2024-09-10T18:18:02.704 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:18:02.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:02 smithi133 ceph-mon[31214]: from='client.24478 v1:172.21.15.133:0/3269090707' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi099=a;smithi133=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:02.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:02 smithi133 ceph-mon[31214]: Saving service node-exporter spec with placement smithi099=a;smithi133=b;count:2 2024-09-10T18:18:02.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:02 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:02.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:02 smithi133 ceph-mon[31214]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s rd, 4.6 KiB/s wr, 142 op/s 2024-09-10T18:18:03.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:02 smithi099 ceph-mon[27037]: from='client.24478 v1:172.21.15.133:0/3269090707' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi099=a;smithi133=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:03.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:02 smithi099 ceph-mon[27037]: Saving service node-exporter spec with placement smithi099=a;smithi133=b;count:2 2024-09-10T18:18:03.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:02 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:03.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:02 smithi099 ceph-mon[27037]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s rd, 4.6 KiB/s wr, 142 op/s 2024-09-10T18:18:03.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:02 smithi099 ceph-mon[37557]: from='client.24478 v1:172.21.15.133:0/3269090707' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi099=a;smithi133=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:03.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:02 smithi099 ceph-mon[37557]: Saving service node-exporter spec with placement smithi099=a;smithi133=b;count:2 2024-09-10T18:18:03.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:02 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:03.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:02 smithi099 ceph-mon[37557]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s rd, 4.6 KiB/s wr, 142 op/s 2024-09-10T18:18:05.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:04 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:05.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:04 smithi133 ceph-mon[31214]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 52 KiB/s rd, 3.9 KiB/s wr, 126 op/s 2024-09-10T18:18:05.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:04 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:05.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:04 smithi099 ceph-mon[27037]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 52 KiB/s rd, 3.9 KiB/s wr, 126 op/s 2024-09-10T18:18:05.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:04 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:05.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:04 smithi099 ceph-mon[37557]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 52 KiB/s rd, 3.9 KiB/s wr, 126 op/s 2024-09-10T18:18:05.262 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled alertmanager update... 2024-09-10T18:18:05.889 DEBUG:teuthology.orchestra.run.smithi099:alertmanager.a> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-10T18:18:05.892 INFO:tasks.cephadm:Adding grafana.a on smithi133 2024-09-10T18:18:05.893 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi133=a' 2024-09-10T18:18:06.137 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:18:06.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:05 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:06.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:05 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:06.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:05 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:06.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:05 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:06.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:05 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:06.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:05 smithi133 ceph-mon[31214]: Deploying daemon prometheus.a on smithi133 2024-09-10T18:18:06.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:05 smithi133 ceph-mon[31214]: from='client.24484 v1:172.21.15.133:0/1073454494' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi099=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:06.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:05 smithi133 ceph-mon[31214]: Saving service alertmanager spec with placement smithi099=a;count:1 2024-09-10T18:18:06.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:05 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:06.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:06.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:06.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:06.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:06.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:06.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[27037]: Deploying daemon prometheus.a on smithi133 2024-09-10T18:18:06.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[27037]: from='client.24484 v1:172.21.15.133:0/1073454494' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi099=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:06.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[27037]: Saving service alertmanager spec with placement smithi099=a;count:1 2024-09-10T18:18:06.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:06.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:06.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:06.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:06.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:06.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:06.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[37557]: Deploying daemon prometheus.a on smithi133 2024-09-10T18:18:06.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[37557]: from='client.24484 v1:172.21.15.133:0/1073454494' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi099=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:06.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[37557]: Saving service alertmanager spec with placement smithi099=a;count:1 2024-09-10T18:18:06.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:05 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:06.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:06 smithi133 ceph-mon[31214]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-10T18:18:07.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:06 smithi099 ceph-mon[27037]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-10T18:18:07.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:06 smithi099 ceph-mon[37557]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-10T18:18:09.077 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled grafana update... 2024-09-10T18:18:09.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:08 smithi133 ceph-mon[31214]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 1.1 KiB/s wr, 34 op/s 2024-09-10T18:18:09.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:08 smithi099 ceph-mon[27037]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 1.1 KiB/s wr, 34 op/s 2024-09-10T18:18:09.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:08 smithi099 ceph-mon[37557]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 1.1 KiB/s wr, 34 op/s 2024-09-10T18:18:10.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:10 smithi133 ceph-mon[31214]: from='client.24490 v1:172.21.15.133:0/2095635731' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi133=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:10.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:10 smithi133 ceph-mon[31214]: Saving service grafana spec with placement smithi133=a;count:1 2024-09-10T18:18:10.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:10 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:10.241 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:18:10.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:10 smithi099 ceph-mon[27037]: from='client.24490 v1:172.21.15.133:0/2095635731' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi133=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:10.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:10 smithi099 ceph-mon[27037]: Saving service grafana spec with placement smithi133=a;count:1 2024-09-10T18:18:10.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:10 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:10.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:10 smithi099 ceph-mon[37557]: from='client.24490 v1:172.21.15.133:0/2095635731' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi133=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:18:10.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:10 smithi099 ceph-mon[37557]: Saving service grafana spec with placement smithi133=a;count:1 2024-09-10T18:18:10.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:10 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:10.758 DEBUG:teuthology.orchestra.run.smithi133:grafana.a> sudo journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@grafana.a.service 2024-09-10T18:18:10.761 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T18:18:10.761 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:18:11.003 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:11.331 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:11 smithi099 ceph-mon[27037]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 13 KiB/s rd, 1005 B/s wr, 30 op/s 2024-09-10T18:18:11.332 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:11 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:18:11.332 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:11 smithi099 ceph-mon[37557]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 13 KiB/s rd, 1005 B/s wr, 30 op/s 2024-09-10T18:18:11.332 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:11 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:18:11.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:11 smithi133 ceph-mon[31214]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 13 KiB/s rd, 1005 B/s wr, 30 op/s 2024-09-10T18:18:11.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:11 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:18:12.874 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-10T18:18:12.874 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBkjeBmyAXTMxAAguc/uu1OPFaeizXClvnTVA== 2024-09-10T18:18:13.377 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:13 smithi133 ceph-mon[31214]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 597 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:18:13.377 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:13 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2366239147' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:18:13.377 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:13 smithi133 ceph-mon[31214]: from='client.34493 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:18:13.378 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:13 smithi133 ceph-mon[31214]: from='client.34493 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:18:13.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:13 smithi099 ceph-mon[37557]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 597 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:18:13.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:13 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2366239147' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:18:13.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:13 smithi099 ceph-mon[37557]: from='client.34493 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:18:13.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:13 smithi099 ceph-mon[37557]: from='client.34493 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:18:13.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:13 smithi099 ceph-mon[27037]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 597 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:18:13.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:13 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2366239147' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:18:13.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:13 smithi099 ceph-mon[27037]: from='client.34493 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:18:13.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:13 smithi099 ceph-mon[27037]: from='client.34493 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:18:13.564 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:18:13.564 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T18:18:13.564 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T18:18:13.604 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:18:14.300 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:18:14.649 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:14 smithi133 systemd[1]: Starting Ceph prometheus.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:18:15.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:15 smithi133 ceph-mon[31214]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:18:15.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:15 smithi099 ceph-mon[37557]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:18:15.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:15 smithi099 ceph-mon[27037]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:18:16.055 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:15 smithi133 podman[60162]: 2024-09-10 18:18:15.834441331 +0000 UTC m=+0.018425144 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-10T18:18:16.323 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 podman[60162]: 2024-09-10 18:18:16.189437666 +0000 UTC m=+0.373421478 container create 9f4d94f2b5ede47f2f5430cb8feeb93afc9d4e6253dfb80b956e8834fde99281 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:18:16.741 INFO:teuthology.orchestra.run.smithi133.stdout:[client.1] 2024-09-10T18:18:16.742 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQBojeBmBYPoKxAA9x6dtwFWaEHgPsed6nlAZw== 2024-09-10T18:18:16.991 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 podman[60162]: 2024-09-10 18:18:16.639238826 +0000 UTC m=+0.823222642 container init 9f4d94f2b5ede47f2f5430cb8feeb93afc9d4e6253dfb80b956e8834fde99281 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:18:16.991 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 podman[60162]: 2024-09-10 18:18:16.642198421 +0000 UTC m=+0.826182235 container start 9f4d94f2b5ede47f2f5430cb8feeb93afc9d4e6253dfb80b956e8834fde99281 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:18:16.991 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.667Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:18:16.992 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.667Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:18:16.992 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.667Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi133 (none))" 2024-09-10T18:18:16.992 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.667Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:18:16.992 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.667Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:18:16.992 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.668Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:18:16.992 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.668Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:18:16.992 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.669Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:18:16.992 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.669Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:18:16.992 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.672Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:18:16.992 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.672Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.141µs 2024-09-10T18:18:16.992 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.672Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:18:16.993 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.672Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-10T18:18:16.993 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.672Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=17.319µs wal_replay_duration=221.852µs wbl_replay_duration=104ns total_replay_duration=257.782µs 2024-09-10T18:18:16.993 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.673Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:18:16.993 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.673Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:18:16.993 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.673Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:18:16.993 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.689Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=16.290864ms db_storage=728ns remote_storage=1.023µs web_handler=353ns query_engine=523ns scrape=484.991µs scrape_sd=16.729µs notify=715ns notify_sd=1.274µs rules=15.585415ms tracing=5.795µs 2024-09-10T18:18:16.993 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.689Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:18:16.993 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:18:16.689Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:18:16.993 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 bash[60162]: 9f4d94f2b5ede47f2f5430cb8feeb93afc9d4e6253dfb80b956e8834fde99281 2024-09-10T18:18:16.993 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:18:16 smithi133 systemd[1]: Started Ceph prometheus.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:18:17.341 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:17 smithi133 ceph-mon[31214]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:17.342 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:17 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.133:0/779305637' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:18:17.342 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:17 smithi133 ceph-mon[31214]: from='client.24502 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:18:17.342 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:17 smithi133 ceph-mon[31214]: from='client.24502 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:18:17.342 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:17 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:17.342 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:17 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:17.342 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:17 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:17.342 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:17 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:18:17.343 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T18:18:17.343 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-10T18:18:17.343 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T18:18:17.381 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T18:18:17.381 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T18:18:17.381 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T18:18:17.409 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[27037]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:17.409 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.133:0/779305637' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:18:17.409 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[27037]: from='client.24502 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:18:17.409 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[27037]: from='client.24502 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:18:17.409 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:17.409 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:17.409 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:17.409 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:18:17.409 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[37557]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:17.410 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.133:0/779305637' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:18:17.410 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[37557]: from='client.24502 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:18:17.410 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[37557]: from='client.24502 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:18:17.410 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:17.410 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:17.410 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' 2024-09-10T18:18:17.410 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:18:17.633 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:18.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ignoring --setuser ceph since I am not root 2024-09-10T18:18:18.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ignoring --setgroup ceph since I am not root 2024-09-10T18:18:18.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:17 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:17.958+0000 7f239ed73ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:18:18.198 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ignoring --setuser ceph since I am not root 2024-09-10T18:18:18.198 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ignoring --setgroup ceph since I am not root 2024-09-10T18:18:18.198 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:17 smithi133 ceph-mgr[33092]: -- 172.21.15.133:0/3185803487 <== mon.1 v1:172.21.15.133:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 842113793 0 0) 0x55910160c9c0 con 0x559101618800 2024-09-10T18:18:18.198 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:17.955+0000 7f9856bbaec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:18:18.198 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:18.009+0000 7f9856bbaec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:18:18.198 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:18.073+0000 7f9856bbaec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:18:18.198 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:18.195+0000 7f9856bbaec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:18:18.340 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:18 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:18.016+0000 7f239ed73ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:18:18.340 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:18 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:18.081+0000 7f239ed73ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:18:18.340 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:18 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:18.204+0000 7f239ed73ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:18:18.340 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:18 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:18.338+0000 7f239ed73ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:18:18.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:18.331+0000 7f9856bbaec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:18:18.849 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:18 smithi099 ceph-mon[27037]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:18:18.850 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:18 smithi099 ceph-mon[27037]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:18:18.850 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:18 smithi099 ceph-mon[37557]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:18:18.850 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:18 smithi099 ceph-mon[37557]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:18:18.850 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:18 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:18.594+0000 7f239ed73ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:18:18.850 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:18 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:18.655+0000 7f239ed73ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:18:18.850 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:18 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:18.818+0000 7f239ed73ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:18:18.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:18 smithi133 ceph-mon[31214]: from='mgr.14150 v1:172.21.15.99:0/2937758572' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:18:18.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:18 smithi133 ceph-mon[31214]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:18:18.991 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:18.583+0000 7f9856bbaec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:18:18.991 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:18.642+0000 7f9856bbaec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:18:18.991 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:18.804+0000 7f9856bbaec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:18:19.601 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:18:19.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:19 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/516724865' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:18:19.991 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:19 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:19.642+0000 7f9856bbaec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:18:19.991 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:19 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:19.707+0000 7f9856bbaec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:18:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:19 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/516724865' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:18:20.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:19 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:19.687+0000 7f239ed73ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:18:20.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:19 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:19.752+0000 7f239ed73ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:18:20.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:19 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/516724865' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:18:20.223 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6800","nonce":2565676961}]},"active_addr":"172.21.15.99:6800/2565676961","active_change":"2024-09-10T18:13:33.759312+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24115,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.99:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":3766947829}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":3857318929}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":961068684}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":1856406675}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":4203140319}]}]} 2024-09-10T18:18:20.228 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T18:18:20.228 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T18:18:20.229 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:18:20.518 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:20.700 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:20.409+0000 7f9856bbaec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:18:20.700 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:20.518+0000 7f9856bbaec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:18:20.701 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:20.638+0000 7f9856bbaec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:18:20.755 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:20 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:20.482+0000 7f239ed73ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:18:20.755 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:20 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:20.601+0000 7f239ed73ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:18:20.755 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:20 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:20.722+0000 7f239ed73ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:18:20.990 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:20.698+0000 7f9856bbaec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:18:20.990 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:20.823+0000 7f9856bbaec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:18:20.991 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:20.890+0000 7f9856bbaec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:18:20.991 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:20.947+0000 7f9856bbaec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:18:21.046 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:20 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:20.787+0000 7f239ed73ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:18:21.046 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:20 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:20.916+0000 7f239ed73ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:18:21.046 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:20 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:20.986+0000 7f239ed73ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:18:21.046 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:21 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:21.043+0000 7f239ed73ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:18:21.309 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:21 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:21.178+0000 7f239ed73ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:18:21.310 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:21 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:21.240+0000 7f239ed73ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:18:21.347 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:21.083+0000 7f9856bbaec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:18:21.347 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:21.147+0000 7f9856bbaec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:18:21.347 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:21.210+0000 7f9856bbaec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:18:21.347 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:21.289+0000 7f9856bbaec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:18:21.348 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:21.345+0000 7f9856bbaec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:18:21.706 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:21 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:21.307+0000 7f239ed73ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:18:21.706 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:21 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:21.389+0000 7f239ed73ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:18:21.706 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:21 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:21.446+0000 7f239ed73ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:18:21.706 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:21 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:21.503+0000 7f239ed73ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:18:21.740 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:21.401+0000 7f9856bbaec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:18:22.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-mon[27037]: Standby manager daemon x restarted 2024-09-10T18:18:22.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-mon[27037]: Standby manager daemon x started 2024-09-10T18:18:22.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-mon[27037]: from='mgr.? v1:172.21.15.133:0/1526997073' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:18:22.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-mon[27037]: from='mgr.? v1:172.21.15.133:0/1526997073' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:18:22.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-mon[27037]: from='mgr.? v1:172.21.15.133:0/1526997073' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:18:22.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-mon[27037]: from='mgr.? v1:172.21.15.133:0/1526997073' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:18:22.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-mon[37557]: Standby manager daemon x restarted 2024-09-10T18:18:22.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-mon[37557]: Standby manager daemon x started 2024-09-10T18:18:22.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-mon[37557]: from='mgr.? v1:172.21.15.133:0/1526997073' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:18:22.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-mon[37557]: from='mgr.? v1:172.21.15.133:0/1526997073' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:18:22.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-mon[37557]: from='mgr.? v1:172.21.15.133:0/1526997073' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:18:22.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-mon[37557]: from='mgr.? v1:172.21.15.133:0/1526997073' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:18:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:18:22.450 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":61,"fsid":"e9807d90-6f9f-11ef-bcea-c7b262605968","created":"2024-09-10T18:12:43.787016+0000","modified":"2024-09-10T18:18:22.301117+0000","last_up_change":"2024-09-10T18:17:31.838123+0000","last_in_change":"2024-09-10T18:17:15.270408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:15:42.054570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-10T18:17:46.315662+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-10T18:17:46.545689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:17:48.185123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:17:50.165377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:17:52.243058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"2d91a1a4-657a-4417-a71c-ac9576241c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6801","nonce":24904954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6802","nonce":24904954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6804","nonce":24904954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6803","nonce":24904954}]},"public_addr":"172.21.15.99:6801/24904954","cluster_addr":"172.21.15.99:6802/24904954","heartbeat_back_addr":"172.21.15.99:6804/24904954","heartbeat_front_addr":"172.21.15.99:6803/24904954","state":["exists","up"]},{"osd":1,"uuid":"8d08b46a-4527-4e05-b267-fee4e9a52b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6805","nonce":719640887}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6806","nonce":719640887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6808","nonce":719640887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6807","nonce":719640887}]},"public_addr":"172.21.15.99:6805/719640887","cluster_addr":"172.21.15.99:6806/719640887","heartbeat_back_addr":"172.21.15.99:6808/719640887","heartbeat_front_addr":"172.21.15.99:6807/719640887","state":["exists","up"]},{"osd":2,"uuid":"283d2b7c-557b-435a-a0b0-391352159d7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6809","nonce":592364138}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6810","nonce":592364138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6812","nonce":592364138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6811","nonce":592364138}]},"public_addr":"172.21.15.99:6809/592364138","cluster_addr":"172.21.15.99:6810/592364138","heartbeat_back_addr":"172.21.15.99:6812/592364138","heartbeat_front_addr":"172.21.15.99:6811/592364138","state":["exists","up"]},{"osd":3,"uuid":"842ef8a8-5e78-4c71-83b2-dba621e4234b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6813","nonce":3452474565}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6814","nonce":3452474565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6816","nonce":3452474565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6815","nonce":3452474565}]},"public_addr":"172.21.15.99:6813/3452474565","cluster_addr":"172.21.15.99:6814/3452474565","heartbeat_back_addr":"172.21.15.99:6816/3452474565","heartbeat_front_addr":"172.21.15.99:6815/3452474565","state":["exists","up"]},{"osd":4,"uuid":"86a9c557-6eee-4f19-97f5-5397ae7edd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6800","nonce":81144752}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6801","nonce":81144752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6803","nonce":81144752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6802","nonce":81144752}]},"public_addr":"172.21.15.133:6800/81144752","cluster_addr":"172.21.15.133:6801/81144752","heartbeat_back_addr":"172.21.15.133:6803/81144752","heartbeat_front_addr":"172.21.15.133:6802/81144752","state":["exists","up"]},{"osd":5,"uuid":"0f408970-5a8c-4957-88c3-532f00fe1461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6804","nonce":2255028820}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6805","nonce":2255028820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6807","nonce":2255028820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6806","nonce":2255028820}]},"public_addr":"172.21.15.133:6804/2255028820","cluster_addr":"172.21.15.133:6805/2255028820","heartbeat_back_addr":"172.21.15.133:6807/2255028820","heartbeat_front_addr":"172.21.15.133:6806/2255028820","state":["exists","up"]},{"osd":6,"uuid":"92985eff-4bb3-408e-bc3f-6bcd6f97d50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6808","nonce":2733605656}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6809","nonce":2733605656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6811","nonce":2733605656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6810","nonce":2733605656}]},"public_addr":"172.21.15.133:6808/2733605656","cluster_addr":"172.21.15.133:6809/2733605656","heartbeat_back_addr":"172.21.15.133:6811/2733605656","heartbeat_front_addr":"172.21.15.133:6810/2733605656","state":["exists","up"]},{"osd":7,"uuid":"94ce876c-555c-47a1-9c35-22d82bf77e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6812","nonce":3452975685}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6813","nonce":3452975685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6815","nonce":3452975685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6814","nonce":3452975685}]},"public_addr":"172.21.15.133:6812/3452975685","cluster_addr":"172.21.15.133:6813/3452975685","heartbeat_back_addr":"172.21.15.133:6815/3452975685","heartbeat_front_addr":"172.21.15.133:6814/3452975685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:14:53.163875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:15:16.351745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:15:39.625973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:16:03.018305+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:16:23.952366+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:16:45.610214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:17:07.929453+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:17:29.891109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3766947829":"2024-09-11T18:18:22.301075+0000","172.21.15.99:0/215619363":"2024-09-11T18:13:11.071109+0000","172.21.15.99:6800/2446219887":"2024-09-11T18:13:11.071109+0000","172.21.15.99:0/1856406675":"2024-09-11T18:18:22.301075+0000","172.21.15.99:0/4203140319":"2024-09-11T18:18:22.301075+0000","172.21.15.99:0/1960046719":"2024-09-11T18:13:11.071109+0000","172.21.15.99:0/1512987781":"2024-09-11T18:13:11.071109+0000","172.21.15.99:0/463482381":"2024-09-11T18:13:33.759212+0000","172.21.15.99:6800/1682160141":"2024-09-11T18:13:33.759212+0000","172.21.15.99:0/193334044":"2024-09-11T18:13:33.759212+0000","172.21.15.99:0/1048941956":"2024-09-11T18:13:33.759212+0000","172.21.15.99:0/3857318929":"2024-09-11T18:18:22.301075+0000","172.21.15.99:0/961068684":"2024-09-11T18:18:22.301075+0000","172.21.15.99:6800/2565676961":"2024-09-11T18:18:22.301075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:18:22.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:22 smithi133 ceph-mon[31214]: Standby manager daemon x restarted 2024-09-10T18:18:22.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:22 smithi133 ceph-mon[31214]: Standby manager daemon x started 2024-09-10T18:18:22.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:22 smithi133 ceph-mon[31214]: from='mgr.? v1:172.21.15.133:0/1526997073' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:18:22.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:22 smithi133 ceph-mon[31214]: from='mgr.? v1:172.21.15.133:0/1526997073' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:18:22.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:22 smithi133 ceph-mon[31214]: from='mgr.? v1:172.21.15.133:0/1526997073' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:18:22.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:22 smithi133 ceph-mon[31214]: from='mgr.? v1:172.21.15.133:0/1526997073' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:18:22.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:22.138+0000 7f9856bbaec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:18:22.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: [10/Sep/2024:18:18:22] ENGINE Bus STARTING 2024-09-10T18:18:22.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: CherryPy Checker: 2024-09-10T18:18:22.492 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: The Application mounted at '' has an empty config. 2024-09-10T18:18:22.492 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: 2024-09-10T18:18:22.492 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: [10/Sep/2024:18:18:22] ENGINE Serving on http://:::9283 2024-09-10T18:18:22.492 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: [10/Sep/2024:18:18:22] ENGINE Bus STARTED 2024-09-10T18:18:22.668 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:22.298+0000 7f239ed73ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:18:22.668 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:18:22] ENGINE Bus STARTING 2024-09-10T18:18:22.668 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: CherryPy Checker: 2024-09-10T18:18:22.669 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: The Application mounted at '' has an empty config. 2024-09-10T18:18:22.669 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:22.669 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:18:22] ENGINE Serving on http://:::9283 2024-09-10T18:18:22.669 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:22 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:18:22] ENGINE Bus STARTED 2024-09-10T18:18:23.271 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T18:18:23.272 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:18:23.491 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:18:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-10T18:18:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: Active manager daemon y restarted 2024-09-10T18:18:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: Activating manager daemon y 2024-09-10T18:18:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:18:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: mgrmap e19: y(active, starting, since 0.0107068s), standbys: x 2024-09-10T18:18:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:18:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:23.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:23.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:18:23.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:18:23.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:23.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:18:23.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:18:23.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:18:23.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:18:23.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:18:23.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:18:23.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:18:23.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:18:23.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:18:23.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:18:23.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: Manager daemon y is now available 2024-09-10T18:18:23.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:23.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:23.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:18:23.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:18:23.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:18:23.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:23 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3646738411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:18:23.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-10T18:18:23.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: Active manager daemon y restarted 2024-09-10T18:18:23.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: Activating manager daemon y 2024-09-10T18:18:23.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:18:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: mgrmap e19: y(active, starting, since 0.0107068s), standbys: x 2024-09-10T18:18:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:18:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:18:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:18:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:18:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:18:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:18:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:18:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:18:23.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:18:23.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:18:23.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:18:23.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:18:23.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:18:23.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: Manager daemon y is now available 2024-09-10T18:18:23.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:23.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:23.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:18:23.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:18:23.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:18:23.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3646738411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:18:23.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-10T18:18:23.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: Active manager daemon y restarted 2024-09-10T18:18:23.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: Activating manager daemon y 2024-09-10T18:18:23.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:18:23.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: mgrmap e19: y(active, starting, since 0.0107068s), standbys: x 2024-09-10T18:18:23.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:18:23.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:18:23.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:18:23.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:18:23.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:18:23.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:18:23.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: Manager daemon y is now available 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:18:23.509 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:18:23.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:18:23.510 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3646738411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:18:23.875 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:23.770+0000 7f2341af5640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:18:23] ENGINE Error in HTTPServer.serve 2024-09-10T18:18:23.875 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: Traceback (most recent call last): 2024-09-10T18:18:23.875 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:18:23.875 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: self._connections.run(self.expiration_interval) 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: self._run(expiration_interval) 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: s = self.context.wrap_socket( 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: return self.sslsocket_class._create( 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: self.do_handshake() 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:18:23.876 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: self._sslobj.do_handshake() 2024-09-10T18:18:23.877 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:18:23.877 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:18:24.270 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:24.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:24 smithi133 ceph-mon[31214]: mgrmap e20: y(active, since 1.01374s), standbys: x 2024-09-10T18:18:24.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:24 smithi133 ceph-mon[31214]: [10/Sep/2024:18:18:23] ENGINE Bus STARTING 2024-09-10T18:18:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:24 smithi133 ceph-mon[31214]: [10/Sep/2024:18:18:23] ENGINE Serving on https://172.21.15.99:7150 2024-09-10T18:18:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:24 smithi133 ceph-mon[31214]: [10/Sep/2024:18:18:23] ENGINE Bus STARTED 2024-09-10T18:18:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:24 smithi133 ceph-mon[31214]: [10/Sep/2024:18:18:23] ENGINE Error in HTTPServer.serve 2024-09-10T18:18:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: Traceback (most recent call last): 2024-09-10T18:18:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:18:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:18:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:18:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: self._run(expiration_interval) 2024-09-10T18:18:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:18:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:18:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:18:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:18:24.492 INFO:journalctl@ceph.mon.b.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:18:24.492 INFO:journalctl@ceph.mon.b.smithi133.stdout: s = self.context.wrap_socket( 2024-09-10T18:18:24.492 INFO:journalctl@ceph.mon.b.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:18:24.492 INFO:journalctl@ceph.mon.b.smithi133.stdout: return self.sslsocket_class._create( 2024-09-10T18:18:24.492 INFO:journalctl@ceph.mon.b.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:18:24.492 INFO:journalctl@ceph.mon.b.smithi133.stdout: self.do_handshake() 2024-09-10T18:18:24.492 INFO:journalctl@ceph.mon.b.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:18:24.492 INFO:journalctl@ceph.mon.b.smithi133.stdout: self._sslobj.do_handshake() 2024-09-10T18:18:24.492 INFO:journalctl@ceph.mon.b.smithi133.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:18:24.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:24 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:24.646 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:24 smithi099 ceph-mon[27037]: mgrmap e20: y(active, since 1.01374s), standbys: x 2024-09-10T18:18:24.646 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:24 smithi099 ceph-mon[27037]: [10/Sep/2024:18:18:23] ENGINE Bus STARTING 2024-09-10T18:18:24.646 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:24 smithi099 ceph-mon[27037]: [10/Sep/2024:18:18:23] ENGINE Serving on https://172.21.15.99:7150 2024-09-10T18:18:24.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:24 smithi099 ceph-mon[27037]: [10/Sep/2024:18:18:23] ENGINE Bus STARTED 2024-09-10T18:18:24.647 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:24 smithi099 ceph-mon[27037]: [10/Sep/2024:18:18:23] ENGINE Error in HTTPServer.serve 2024-09-10T18:18:24.647 INFO:journalctl@ceph.mon.a.smithi099.stdout: Traceback (most recent call last): 2024-09-10T18:18:24.647 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:18:24.647 INFO:journalctl@ceph.mon.a.smithi099.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:18:24.647 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:18:24.647 INFO:journalctl@ceph.mon.a.smithi099.stdout: self._run(expiration_interval) 2024-09-10T18:18:24.647 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:18:24.647 INFO:journalctl@ceph.mon.a.smithi099.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:18:24.647 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:18:24.647 INFO:journalctl@ceph.mon.a.smithi099.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:18:24.647 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:18:24.648 INFO:journalctl@ceph.mon.a.smithi099.stdout: s = self.context.wrap_socket( 2024-09-10T18:18:24.648 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:18:24.648 INFO:journalctl@ceph.mon.a.smithi099.stdout: return self.sslsocket_class._create( 2024-09-10T18:18:24.648 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:18:24.648 INFO:journalctl@ceph.mon.a.smithi099.stdout: self.do_handshake() 2024-09-10T18:18:24.648 INFO:journalctl@ceph.mon.a.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:18:24.648 INFO:journalctl@ceph.mon.a.smithi099.stdout: self._sslobj.do_handshake() 2024-09-10T18:18:24.648 INFO:journalctl@ceph.mon.a.smithi099.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:18:24.648 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:24 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:24.648 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:24 smithi099 ceph-mon[37557]: mgrmap e20: y(active, since 1.01374s), standbys: x 2024-09-10T18:18:24.649 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:24 smithi099 ceph-mon[37557]: [10/Sep/2024:18:18:23] ENGINE Bus STARTING 2024-09-10T18:18:24.649 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:24 smithi099 ceph-mon[37557]: [10/Sep/2024:18:18:23] ENGINE Serving on https://172.21.15.99:7150 2024-09-10T18:18:24.649 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:24 smithi099 ceph-mon[37557]: [10/Sep/2024:18:18:23] ENGINE Bus STARTED 2024-09-10T18:18:24.649 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:24 smithi099 ceph-mon[37557]: [10/Sep/2024:18:18:23] ENGINE Error in HTTPServer.serve 2024-09-10T18:18:24.649 INFO:journalctl@ceph.mon.c.smithi099.stdout: Traceback (most recent call last): 2024-09-10T18:18:24.649 INFO:journalctl@ceph.mon.c.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:18:24.649 INFO:journalctl@ceph.mon.c.smithi099.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:18:24.649 INFO:journalctl@ceph.mon.c.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:18:24.649 INFO:journalctl@ceph.mon.c.smithi099.stdout: self._run(expiration_interval) 2024-09-10T18:18:24.649 INFO:journalctl@ceph.mon.c.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:18:24.649 INFO:journalctl@ceph.mon.c.smithi099.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:18:24.649 INFO:journalctl@ceph.mon.c.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:18:24.650 INFO:journalctl@ceph.mon.c.smithi099.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:18:24.650 INFO:journalctl@ceph.mon.c.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:18:24.650 INFO:journalctl@ceph.mon.c.smithi099.stdout: s = self.context.wrap_socket( 2024-09-10T18:18:24.650 INFO:journalctl@ceph.mon.c.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:18:24.650 INFO:journalctl@ceph.mon.c.smithi099.stdout: return self.sslsocket_class._create( 2024-09-10T18:18:24.650 INFO:journalctl@ceph.mon.c.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:18:24.650 INFO:journalctl@ceph.mon.c.smithi099.stdout: self.do_handshake() 2024-09-10T18:18:24.650 INFO:journalctl@ceph.mon.c.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:18:24.650 INFO:journalctl@ceph.mon.c.smithi099.stdout: self._sslobj.do_handshake() 2024-09-10T18:18:24.650 INFO:journalctl@ceph.mon.c.smithi099.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:18:24.650 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:24 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.130 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:18:26.130 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":61,"fsid":"e9807d90-6f9f-11ef-bcea-c7b262605968","created":"2024-09-10T18:12:43.787016+0000","modified":"2024-09-10T18:18:22.301117+0000","last_up_change":"2024-09-10T18:17:31.838123+0000","last_in_change":"2024-09-10T18:17:15.270408+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:15:42.054570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-10T18:17:46.315662+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-10T18:17:46.545689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:17:48.185123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:17:50.165377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:17:52.243058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"2d91a1a4-657a-4417-a71c-ac9576241c4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6801","nonce":24904954}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6802","nonce":24904954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6804","nonce":24904954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6803","nonce":24904954}]},"public_addr":"172.21.15.99:6801/24904954","cluster_addr":"172.21.15.99:6802/24904954","heartbeat_back_addr":"172.21.15.99:6804/24904954","heartbeat_front_addr":"172.21.15.99:6803/24904954","state":["exists","up"]},{"osd":1,"uuid":"8d08b46a-4527-4e05-b267-fee4e9a52b38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6805","nonce":719640887}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6806","nonce":719640887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6808","nonce":719640887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6807","nonce":719640887}]},"public_addr":"172.21.15.99:6805/719640887","cluster_addr":"172.21.15.99:6806/719640887","heartbeat_back_addr":"172.21.15.99:6808/719640887","heartbeat_front_addr":"172.21.15.99:6807/719640887","state":["exists","up"]},{"osd":2,"uuid":"283d2b7c-557b-435a-a0b0-391352159d7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6809","nonce":592364138}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6810","nonce":592364138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6812","nonce":592364138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6811","nonce":592364138}]},"public_addr":"172.21.15.99:6809/592364138","cluster_addr":"172.21.15.99:6810/592364138","heartbeat_back_addr":"172.21.15.99:6812/592364138","heartbeat_front_addr":"172.21.15.99:6811/592364138","state":["exists","up"]},{"osd":3,"uuid":"842ef8a8-5e78-4c71-83b2-dba621e4234b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6813","nonce":3452474565}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6814","nonce":3452474565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6816","nonce":3452474565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.99:6815","nonce":3452474565}]},"public_addr":"172.21.15.99:6813/3452474565","cluster_addr":"172.21.15.99:6814/3452474565","heartbeat_back_addr":"172.21.15.99:6816/3452474565","heartbeat_front_addr":"172.21.15.99:6815/3452474565","state":["exists","up"]},{"osd":4,"uuid":"86a9c557-6eee-4f19-97f5-5397ae7edd50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6800","nonce":81144752}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6801","nonce":81144752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6803","nonce":81144752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6802","nonce":81144752}]},"public_addr":"172.21.15.133:6800/81144752","cluster_addr":"172.21.15.133:6801/81144752","heartbeat_back_addr":"172.21.15.133:6803/81144752","heartbeat_front_addr":"172.21.15.133:6802/81144752","state":["exists","up"]},{"osd":5,"uuid":"0f408970-5a8c-4957-88c3-532f00fe1461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6804","nonce":2255028820}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6805","nonce":2255028820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6807","nonce":2255028820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6806","nonce":2255028820}]},"public_addr":"172.21.15.133:6804/2255028820","cluster_addr":"172.21.15.133:6805/2255028820","heartbeat_back_addr":"172.21.15.133:6807/2255028820","heartbeat_front_addr":"172.21.15.133:6806/2255028820","state":["exists","up"]},{"osd":6,"uuid":"92985eff-4bb3-408e-bc3f-6bcd6f97d50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6808","nonce":2733605656}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6809","nonce":2733605656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6811","nonce":2733605656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6810","nonce":2733605656}]},"public_addr":"172.21.15.133:6808/2733605656","cluster_addr":"172.21.15.133:6809/2733605656","heartbeat_back_addr":"172.21.15.133:6811/2733605656","heartbeat_front_addr":"172.21.15.133:6810/2733605656","state":["exists","up"]},{"osd":7,"uuid":"94ce876c-555c-47a1-9c35-22d82bf77e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6812","nonce":3452975685}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6813","nonce":3452975685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6815","nonce":3452975685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.133:6814","nonce":3452975685}]},"public_addr":"172.21.15.133:6812/3452975685","cluster_addr":"172.21.15.133:6813/3452975685","heartbeat_back_addr":"172.21.15.133:6815/3452975685","heartbeat_front_addr":"172.21.15.133:6814/3452975685","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:14:53.163875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:15:16.351745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:15:39.625973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:16:03.018305+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:16:23.952366+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:16:45.610214+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:17:07.929453+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:17:29.891109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:0/3766947829":"2024-09-11T18:18:22.301075+0000","172.21.15.99:0/215619363":"2024-09-11T18:13:11.071109+0000","172.21.15.99:6800/2446219887":"2024-09-11T18:13:11.071109+0000","172.21.15.99:0/1856406675":"2024-09-11T18:18:22.301075+0000","172.21.15.99:0/4203140319":"2024-09-11T18:18:22.301075+0000","172.21.15.99:0/1960046719":"2024-09-11T18:13:11.071109+0000","172.21.15.99:0/1512987781":"2024-09-11T18:13:11.071109+0000","172.21.15.99:0/463482381":"2024-09-11T18:13:33.759212+0000","172.21.15.99:6800/1682160141":"2024-09-11T18:13:33.759212+0000","172.21.15.99:0/193334044":"2024-09-11T18:13:33.759212+0000","172.21.15.99:0/1048941956":"2024-09-11T18:13:33.759212+0000","172.21.15.99:0/3857318929":"2024-09-11T18:18:22.301075+0000","172.21.15.99:0/961068684":"2024-09-11T18:18:22.301075+0000","172.21.15.99:6800/2565676961":"2024-09-11T18:18:22.301075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:18:26.130 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:25 smithi099 ceph-mon[37557]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:18:26.130 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:25 smithi099 ceph-mon[37557]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-10T18:18:26.131 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:25 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.131 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:25 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.131 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:25 smithi099 ceph-mon[27037]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:18:26.131 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:25 smithi099 ceph-mon[27037]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-10T18:18:26.131 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:25 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.131 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:25 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:25 smithi133 ceph-mon[31214]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:18:26.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:25 smithi133 ceph-mon[31214]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-10T18:18:26.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:25 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:25 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.866 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T18:18:26.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T18:18:26.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T18:18:26.868 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T18:18:26.868 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T18:18:26.869 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T18:18:26.869 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T18:18:26.869 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T18:18:26.906 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.906 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.906 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:26.906 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:26.906 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:26.906 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:26.906 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[27037]: Adjusting osd_memory_target on smithi133 to 3760M 2024-09-10T18:18:26.906 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.906 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.907 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.907 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3748232788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:18:26.907 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.907 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.907 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:26.907 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:26.907 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:26.907 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:26.907 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[37557]: Adjusting osd_memory_target on smithi133 to 3760M 2024-09-10T18:18:26.908 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.908 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.908 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:26.908 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:26 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3748232788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:18:27.172 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:26 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:27.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:26 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:27.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:26 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:27.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:26 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:27.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:26 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:27.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:26 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:27.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:26 smithi133 ceph-mon[31214]: Adjusting osd_memory_target on smithi133 to 3760M 2024-09-10T18:18:27.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:26 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:27.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:26 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:27.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:26 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:27.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:26 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3748232788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:18:27.347 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:27.406 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:27.407 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:27.423 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:18:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:18:27.426 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:27.426 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:27.766 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:27.767 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:28.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[27037]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:18:28.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:28.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:28.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:28.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:28.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:28.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:28.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:28.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:28.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:28.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[27037]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-10T18:18:28.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[37557]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:18:28.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:28.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:28.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:28.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:28.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:28.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:28.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:28.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:28.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:28.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:27 smithi099 ceph-mon[37557]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-10T18:18:28.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:27 smithi133 ceph-mon[31214]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:18:28.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:27 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:28.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:27 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:28.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:27 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:28.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:27 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:28.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:27 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:28.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:27 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:18:28.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:27 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:28.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:27 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:18:28.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:27 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:18:28.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:27 smithi133 ceph-mon[31214]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-10T18:18:29.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[27037]: Adjusting osd_memory_target on smithi099 to 3760M 2024-09-10T18:18:29.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[27037]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-10T18:18:29.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[27037]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T18:18:29.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[27037]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:18:29.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[27037]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:18:29.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[27037]: Updating smithi099:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:18:29.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[27037]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:18:29.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[27037]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:18:29.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[37557]: Adjusting osd_memory_target on smithi099 to 3760M 2024-09-10T18:18:29.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[37557]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-10T18:18:29.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[37557]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T18:18:29.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[37557]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:18:29.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[37557]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:18:29.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[37557]: Updating smithi099:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:18:29.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[37557]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:18:29.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[37557]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:18:29.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.008 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:28 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:28 smithi133 ceph-mon[31214]: Adjusting osd_memory_target on smithi099 to 3760M 2024-09-10T18:18:29.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:28 smithi133 ceph-mon[31214]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-10T18:18:29.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:28 smithi133 ceph-mon[31214]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T18:18:29.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:28 smithi133 ceph-mon[31214]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:18:29.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:28 smithi133 ceph-mon[31214]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:18:29.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:28 smithi133 ceph-mon[31214]: Updating smithi099:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:18:29.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:28 smithi133 ceph-mon[31214]: Updating smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:18:29.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:28 smithi133 ceph-mon[31214]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:18:29.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:28 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:28 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:28 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:28 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:29.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:28 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:30.127 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:29 smithi099 ceph-mon[27037]: Updating smithi099:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:18:30.127 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:29 smithi099 ceph-mon[27037]: Deploying daemon alertmanager.a on smithi099 2024-09-10T18:18:30.127 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:29 smithi099 ceph-mon[27037]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:18:30.128 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:29 smithi099 ceph-mon[37557]: Updating smithi099:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:18:30.128 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:29 smithi099 ceph-mon[37557]: Deploying daemon alertmanager.a on smithi099 2024-09-10T18:18:30.128 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:29 smithi099 ceph-mon[37557]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:18:30.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:29 smithi133 ceph-mon[31214]: Updating smithi099:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:18:30.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:29 smithi133 ceph-mon[31214]: Deploying daemon alertmanager.a on smithi099 2024-09-10T18:18:30.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:29 smithi133 ceph-mon[31214]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:18:32.090 INFO:teuthology.orchestra.run.smithi099.stdout:180388626450 2024-09-10T18:18:32.091 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T18:18:32.175 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:31 smithi099 ceph-mon[27037]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:18:32.176 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:31 smithi099 ceph-mon[37557]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:18:32.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:31 smithi133 ceph-mon[31214]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:18:33.197 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:32 smithi099 ceph-mon[27037]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:18:33.197 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:32 smithi099 ceph-mon[37557]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:18:33.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:32 smithi133 ceph-mon[31214]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:18:33.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:33 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:18:34.213 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:33 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:18:34.213 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:33 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:18:34.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:33 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:18:35.156 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:35.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:34 smithi133 ceph-mon[31214]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:18:35.253 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:34 smithi099 ceph-mon[27037]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:18:35.253 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:34 smithi099 ceph-mon[37557]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:18:37.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:37 smithi133 ceph-mon[31214]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:18:37.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:37 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:18:37.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:18:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:18:37.628 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:37 smithi099 ceph-mon[27037]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:18:37.629 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:37 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:18:37.629 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:37 smithi099 ceph-mon[37557]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:18:37.629 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:37 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:18:38.304 INFO:teuthology.orchestra.run.smithi099.stdout:206158430222 2024-09-10T18:18:38.305 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T18:18:39.434 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:39 smithi099 ceph-mon[27037]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:18:39.435 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:39 smithi099 ceph-mon[37557]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:18:39.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:39 smithi133 ceph-mon[31214]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:18:39.746 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:40.613 INFO:teuthology.orchestra.run.smithi099.stdout:55834574890 2024-09-10T18:18:40.613 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T18:18:41.540 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:41 smithi099 ceph-mon[27037]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:18:41.540 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:41 smithi099 ceph-mon[37557]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:18:41.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:41 smithi133 ceph-mon[31214]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:18:42.400 INFO:teuthology.orchestra.run.smithi099.stdout:154618822680 2024-09-10T18:18:42.400 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T18:18:43.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:43 smithi133 ceph-mon[31214]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:43.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:43 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:18:43.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:43 smithi099 ceph-mon[27037]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:43.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:43 smithi099 ceph-mon[37557]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:44.658 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:44 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:18:44.658 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:44 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:18:44.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:44 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:18:45.106 INFO:teuthology.orchestra.run.smithi099.stdout:128849018908 2024-09-10T18:18:45.106 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T18:18:45.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:45 smithi133 ceph-mon[31214]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:18:45.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:45 smithi099 ceph-mon[27037]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:18:45.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:45 smithi099 ceph-mon[37557]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:18:46.825 INFO:teuthology.orchestra.run.smithi099.stdout:77309411366 2024-09-10T18:18:46.825 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T18:18:47.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:47 smithi133 ceph-mon[31214]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:47.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:47 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:18:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:18:47.604 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:47 smithi099 ceph-mon[27037]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:47.604 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:47 smithi099 ceph-mon[37557]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:47.758 INFO:teuthology.orchestra.run.smithi099.stdout:180388626453 2024-09-10T18:18:48.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:48 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1583959955' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:18:48.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:48 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1583959955' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:18:48.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:48 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1583959955' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:18:48.766 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:48.767 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:48.782 INFO:teuthology.orchestra.run.smithi099.stdout:107374182434 2024-09-10T18:18:48.783 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T18:18:49.497 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:49 smithi099 ceph-mon[27037]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:18:49.498 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:49 smithi099 ceph-mon[37557]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:18:49.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:49 smithi133 ceph-mon[31214]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:18:51.525 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:51 smithi099 ceph-mon[27037]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:51.526 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:51 smithi099 ceph-mon[37557]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:51.698 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:51.698 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:51.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:51 smithi133 ceph-mon[31214]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:52.118 INFO:teuthology.orchestra.run.smithi099.stdout:34359738416 2024-09-10T18:18:52.118 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T18:18:52.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:52 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:18:52.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:52 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:18:52.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:52 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:18:53.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:53 smithi099 ceph-mon[27037]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:53.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:53 smithi099 ceph-mon[37557]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:53.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:53 smithi133 ceph-mon[31214]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:53.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:18:53 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:18:54.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:54 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:18:54.739 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:54 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:18:54.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:54 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:18:55.671 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:55 smithi099 ceph-mon[27037]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:18:55.671 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:55 smithi099 ceph-mon[37557]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:18:55.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:55 smithi133 ceph-mon[31214]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:18:55.958 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626450 got 180388626453 for osd.6 2024-09-10T18:18:55.958 DEBUG:teuthology.parallel:result is None 2024-09-10T18:18:56.255 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:56 smithi099 podman[70321]: 2024-09-10 18:18:56.137163415 +0000 UTC m=+2.323819614 container create c4e823da713ac206541c82ae0d3da3cbedf95dafaad8508c400cc912d7bc5e26 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:18:56.694 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:56 smithi099 podman[70321]: 2024-09-10 18:18:56.465374635 +0000 UTC m=+2.652030787 container init c4e823da713ac206541c82ae0d3da3cbedf95dafaad8508c400cc912d7bc5e26 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:18:56.694 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:56 smithi099 podman[70321]: 2024-09-10 18:18:56.46893704 +0000 UTC m=+2.655593200 container start c4e823da713ac206541c82ae0d3da3cbedf95dafaad8508c400cc912d7bc5e26 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:18:56.694 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:56 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[70410]: ts=2024-09-10T18:18:56.484Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T18:18:56.694 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:56 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[70410]: ts=2024-09-10T18:18:56.484Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T18:18:56.695 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:56 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[70410]: ts=2024-09-10T18:18:56.485Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.99 port=9094 2024-09-10T18:18:56.695 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:56 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[70410]: ts=2024-09-10T18:18:56.486Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T18:18:56.695 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:56 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[70410]: ts=2024-09-10T18:18:56.510Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:18:56.695 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:56 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[70410]: ts=2024-09-10T18:18:56.510Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:18:56.695 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:56 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[70410]: ts=2024-09-10T18:18:56.514Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T18:18:56.695 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:56 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[70410]: ts=2024-09-10T18:18:56.514Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T18:18:56.695 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:56 smithi099 bash[70321]: c4e823da713ac206541c82ae0d3da3cbedf95dafaad8508c400cc912d7bc5e26 2024-09-10T18:18:56.695 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:56 smithi099 systemd[1]: Started Ceph alertmanager.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:18:56.977 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:56 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:18:56] ENGINE Bus STOPPING 2024-09-10T18:18:57.255 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:18:57] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:18:57.255 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:18:57] ENGINE Bus STOPPED 2024-09-10T18:18:57.255 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:18:57] ENGINE Bus STARTING 2024-09-10T18:18:57.255 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:18:57] ENGINE Serving on http://:::9283 2024-09-10T18:18:57.255 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:18:57] ENGINE Bus STARTED 2024-09-10T18:18:57.473 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:57.474 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:57.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:18:57 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:18:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:18:57.513 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:18:57.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[27037]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:57.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:18:57.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[27037]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:18:57.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[27037]: Deploying daemon grafana.a on smithi133 2024-09-10T18:18:57.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[37557]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:57.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:18:57.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[37557]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:18:57.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[37557]: Deploying daemon grafana.a on smithi133 2024-09-10T18:18:57.758 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:57 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:57 smithi133 ceph-mon[31214]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:18:57.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:57 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:57 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:57 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:57 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:57 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:57 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.992 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:57 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:18:57.992 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:57 smithi133 ceph-mon[31214]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:18:57.992 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:57 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:57.992 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:57 smithi133 ceph-mon[31214]: Deploying daemon grafana.a on smithi133 2024-09-10T18:18:57.992 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:57 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:18:58.755 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:18:58 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[70410]: ts=2024-09-10T18:18:58.486Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000315791s 2024-09-10T18:18:59.574 INFO:teuthology.orchestra.run.smithi099.stdout:107374182437 2024-09-10T18:18:59.774 INFO:teuthology.orchestra.run.smithi099.stdout:206158430227 2024-09-10T18:18:59.864 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:59 smithi099 ceph-mon[27037]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:18:59.865 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:18:59 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1212641965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:18:59.865 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:59 smithi099 ceph-mon[37557]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:18:59.865 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:18:59 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1212641965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:18:59.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:59 smithi133 ceph-mon[31214]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:18:59.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:18:59 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1212641965' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:19:00.726 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430222 got 206158430227 for osd.7 2024-09-10T18:19:00.727 DEBUG:teuthology.parallel:result is None 2024-09-10T18:19:00.934 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:00 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/886577287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:19:00.934 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:00 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/886577287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:19:00.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:00 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/886577287' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:19:01.954 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:01 smithi099 ceph-mon[27037]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:01.954 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:01 smithi099 ceph-mon[37557]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:01.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:01 smithi133 ceph-mon[31214]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:02.947 INFO:teuthology.orchestra.run.smithi099.stdout:34359738418 2024-09-10T18:19:03.139 INFO:teuthology.orchestra.run.smithi099.stdout:77309411371 2024-09-10T18:19:03.672 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:19:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:19:03.695 INFO:teuthology.orchestra.run.smithi099.stdout:55834574894 2024-09-10T18:19:03.706 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:03 smithi099 ceph-mon[27037]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:03.706 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:03 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3074392769' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:19:03.706 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:03 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3646453017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:19:03.706 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:03 smithi099 ceph-mon[37557]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:03.706 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:03 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3074392769' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:19:03.706 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:03 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3646453017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:19:03.942 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411366 got 77309411371 for osd.2 2024-09-10T18:19:03.942 DEBUG:teuthology.parallel:result is None 2024-09-10T18:19:03.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:03 smithi133 ceph-mon[31214]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:03.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:03 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3074392769' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:19:03.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:03 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3646453017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:19:04.488 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182434 got 107374182437 for osd.3 2024-09-10T18:19:04.488 DEBUG:teuthology.parallel:result is None 2024-09-10T18:19:04.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:04 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:04.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:04 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2602025033' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:19:04.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:04 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:04.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:04 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2602025033' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:19:04.881 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738416 got 34359738418 for osd.0 2024-09-10T18:19:04.882 DEBUG:teuthology.parallel:result is None 2024-09-10T18:19:04.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:04 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:04.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:04 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2602025033' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:19:05.926 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:05 smithi099 ceph-mon[27037]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:05.926 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:05 smithi099 ceph-mon[37557]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:05.944 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:05 smithi133 ceph-mon[31214]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:06.172 INFO:teuthology.orchestra.run.smithi099.stdout:154618822686 2024-09-10T18:19:06.391 INFO:teuthology.orchestra.run.smithi099.stdout:128849018913 2024-09-10T18:19:06.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:06 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/726081833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:19:06.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:06 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/613783652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:19:06.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:06 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/726081833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:19:06.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:06 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/613783652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:19:06.756 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:06 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[70410]: ts=2024-09-10T18:19:06.488Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001974158s 2024-09-10T18:19:06.806 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822680 got 154618822686 for osd.5 2024-09-10T18:19:06.806 DEBUG:teuthology.parallel:result is None 2024-09-10T18:19:06.947 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:06 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/726081833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:19:06.947 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:06 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/613783652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:19:07.206 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574890 got 55834574894 for osd.1 2024-09-10T18:19:07.206 DEBUG:teuthology.parallel:result is None 2024-09-10T18:19:07.240 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:19:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:19:07.241 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 systemd[1]: Starting Ceph grafana.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:19:07.684 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 podman[62066]: 2024-09-10 18:19:07.286065976 +0000 UTC m=+0.014968759 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-10T18:19:07.684 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 podman[62066]: 2024-09-10 18:19:07.397295734 +0000 UTC m=+0.126198519 container create 159ca1ecb4a7e339c2ebbe520fd13887b558a9f105ae7a98352dd8e043e2a0cd (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a, io.openshift.expose-services=, com.redhat.component=ubi8-container, release=239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.tags=base rhel8, architecture=x86_64, name=ubi8, vendor=Red Hat, Inc., distribution-scope=public, build-date=2022-04-29T11:48:11.055897, maintainer=Paul Cuzner , com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-09-10T18:19:07.684 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 podman[62066]: 2024-09-10 18:19:07.624673019 +0000 UTC m=+0.353575809 container init 159ca1ecb4a7e339c2ebbe520fd13887b558a9f105ae7a98352dd8e043e2a0cd (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a, vcs-type=git, vendor=Red Hat, Inc., io.openshift.expose-services=, architecture=x86_64, io.openshift.tags=base rhel8, io.buildah.version=1.32.0, name=ubi8, version=8.5, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, release=239.1651231664, maintainer=Paul Cuzner , io.k8s.display-name=Red Hat Universal Base Image 8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, build-date=2022-04-29T11:48:11.055897, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container) 2024-09-10T18:19:07.684 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 podman[62066]: 2024-09-10 18:19:07.62750368 +0000 UTC m=+0.356406465 container start 159ca1ecb4a7e339c2ebbe520fd13887b558a9f105ae7a98352dd8e043e2a0cd (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a, io.openshift.expose-services=, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, maintainer=Paul Cuzner , name=ubi8, version=8.5, build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., architecture=x86_64, io.buildah.version=1.32.0, vcs-type=git, distribution-scope=public, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, com.redhat.component=ubi8-container, io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-09-10T18:19:07.914 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:07 smithi099 ceph-mon[27037]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:07.915 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:07 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:19:07.915 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:07 smithi099 ceph-mon[37557]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:07.915 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:07 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:19:07.916 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018908 got 128849018913 for osd.4 2024-09-10T18:19:07.916 DEBUG:teuthology.parallel:result is None 2024-09-10T18:19:07.917 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T18:19:07.917 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:19:07.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-mon[31214]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:07.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:19:07.935 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 bash[62066]: 159ca1ecb4a7e339c2ebbe520fd13887b558a9f105ae7a98352dd8e043e2a0cd 2024-09-10T18:19:07.935 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 systemd[1]: Started Ceph grafana.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:19:07.935 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.786623803Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-10T18:19:07.935 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.786841324Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-10T18:19:07.935 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.786866156Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-10T18:19:07.935 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.786879087Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-10T18:19:07.935 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.786889507Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-10T18:19:07.936 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.786900529Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-10T18:19:07.936 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.786910777Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-10T18:19:07.936 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.786947056Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-10T18:19:07.936 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.786958203Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-10T18:19:07.936 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.786968705Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-10T18:19:07.936 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.786979217Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-10T18:19:07.936 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.786989313Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-10T18:19:07.936 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.786999683Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-10T18:19:07.936 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=settings t=2024-09-10T18:19:07.787010076Z level=info msg="App mode production" 2024-09-10T18:19:07.937 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=sqlstore t=2024-09-10T18:19:07.787062992Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-10T18:19:07.937 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=sqlstore t=2024-09-10T18:19:07.787085591Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-10T18:19:07.937 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.787937509Z level=info msg="Starting DB migrations" 2024-09-10T18:19:07.937 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.788021009Z level=info msg="Executing migration" id="create migration_log table" 2024-09-10T18:19:07.937 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.788650037Z level=info msg="Executing migration" id="create user table" 2024-09-10T18:19:07.937 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.789131595Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-10T18:19:07.937 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.789643089Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-10T18:19:07.937 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.792498006Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-10T18:19:07.937 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.795222938Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-10T18:19:07.937 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.798119029Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-10T18:19:07.937 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.800900305Z level=info msg="Executing migration" id="create user table v2" 2024-09-10T18:19:07.937 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.803868297Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-10T18:19:07.938 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.806483894Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-10T18:19:07.938 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.80941887Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-10T18:19:07.938 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.811985784Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-10T18:19:07.938 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.814766531Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-10T18:19:07.938 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.815221598Z level=info msg="Executing migration" id="Update user table charset" 2024-09-10T18:19:07.938 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.81812343Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-10T18:19:07.938 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.82084136Z level=info msg="Executing migration" id="Add missing user data" 2024-09-10T18:19:07.938 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.823338052Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-10T18:19:07.938 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.82379336Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-10T18:19:07.939 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.826684468Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-10T18:19:07.939 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.829249863Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-10T18:19:07.940 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.832151812Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-10T18:19:07.940 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.832666572Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-10T18:19:07.940 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.835378178Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-10T18:19:07.940 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.838122317Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-10T18:19:07.940 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.838538173Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-10T18:19:07.940 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.841332444Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-10T18:19:07.940 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.844058114Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-10T18:19:07.940 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.845008391Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-10T18:19:07.940 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.845693918Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-10T18:19:07.940 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.846525376Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-10T18:19:07.940 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.847548702Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-10T18:19:07.941 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.848327599Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-10T18:19:07.941 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.849132098Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-10T18:19:07.941 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.849994225Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-10T18:19:07.941 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.850834755Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-10T18:19:07.941 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.851679054Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-10T18:19:07.941 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.852656503Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-10T18:19:07.941 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.853359007Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-10T18:19:07.941 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.853746343Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-10T18:19:07.941 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.85466834Z level=info msg="Executing migration" id="create star table" 2024-09-10T18:19:07.941 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.855592324Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-10T18:19:07.941 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.856362779Z level=info msg="Executing migration" id="create org table v1" 2024-09-10T18:19:07.942 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.856785332Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-10T18:19:07.942 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.85765574Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-10T18:19:07.942 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.858503171Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-10T18:19:07.942 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.858959958Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-10T18:19:07.942 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.859907196Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-10T18:19:07.942 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.860627547Z level=info msg="Executing migration" id="Update org table charset" 2024-09-10T18:19:07.942 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.861458342Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-10T18:19:07.942 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.861804562Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-10T18:19:07.942 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.86214603Z level=info msg="Executing migration" id="create dashboard table" 2024-09-10T18:19:07.942 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.862987014Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-10T18:19:07.942 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.863971368Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-10T18:19:07.942 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.864756887Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-10T18:19:07.943 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.8653643Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-10T18:19:07.943 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.866272978Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-10T18:19:07.943 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.867111484Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-10T18:19:07.943 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.867982528Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-10T18:19:07.943 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.868385589Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-10T18:19:07.943 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.869243153Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-10T18:19:07.943 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.870080319Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-10T18:19:07.943 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.870998572Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-10T18:19:07.943 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.8714493Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-10T18:19:07.943 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.874253219Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-10T18:19:07.943 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.874779039Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-10T18:19:07.943 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.877085599Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-10T18:19:07.944 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.880069017Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-10T18:19:07.944 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.880516819Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-10T18:19:07.944 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.88333289Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-10T18:19:07.944 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.88379632Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-10T18:19:07.944 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.886137133Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-10T18:19:07.944 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.88886936Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-10T18:19:07.944 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.889219456Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-10T18:19:07.944 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.892202456Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-10T18:19:07.944 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.895031817Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-10T18:19:07.944 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.897983282Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-10T18:19:07.944 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.89856188Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-10T18:19:07.944 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.90136465Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-10T18:19:07.945 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.901822702Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-10T18:19:07.945 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.904318688Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-10T18:19:07.945 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.907048444Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-10T18:19:07.945 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.907552101Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-10T18:19:07.945 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.910399231Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-10T18:19:07.945 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.913264162Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-10T18:19:07.945 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.915821196Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-10T18:19:07.945 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.916396057Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-10T18:19:07.946 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.918997335Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-10T18:19:07.946 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.921766412Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-10T18:19:07.946 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.924408666Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-10T18:19:07.946 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.924944064Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-10T18:19:07.946 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.927632187Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-10T18:19:07.946 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.930549565Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-10T18:19:07.947 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.933253392Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-10T18:19:07.947 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.933805006Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-10T18:19:08.185 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.936627108Z level=info msg="Executing migration" id="create data_source table" 2024-09-10T18:19:08.185 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.939241346Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-10T18:19:08.185 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.942036923Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-10T18:19:08.185 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.942557724Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-10T18:19:08.185 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.943359586Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-10T18:19:08.186 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.943773727Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-10T18:19:08.186 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.944794826Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-10T18:19:08.186 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.9456266Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-10T18:19:08.186 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.946054711Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-10T18:19:08.186 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.946860565Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-10T18:19:08.186 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.947766874Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-10T18:19:08.186 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.948453972Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-10T18:19:08.186 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.948945007Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-10T18:19:08.186 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.949236647Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-10T18:19:08.186 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.949709933Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-10T18:19:08.186 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.950256559Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-10T18:19:08.186 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.950581634Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-10T18:19:08.187 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.951440733Z level=info msg="Executing migration" id="Add uid column" 2024-09-10T18:19:08.187 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.952174026Z level=info msg="Executing migration" id="Update uid value" 2024-09-10T18:19:08.188 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.952488716Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-10T18:19:08.188 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.953248748Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-10T18:19:08.188 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.953759505Z level=info msg="Executing migration" id="create api_key table" 2024-09-10T18:19:08.188 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.954205421Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-10T18:19:08.188 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.95502235Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-10T18:19:08.188 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.955595677Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-10T18:19:08.188 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.956469125Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-10T18:19:08.188 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.957230235Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-10T18:19:08.188 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.958073408Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-10T18:19:08.188 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.958935869Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-10T18:19:08.188 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.960134738Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-10T18:19:08.189 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.96090185Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-10T18:19:08.189 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.961346649Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-10T18:19:08.189 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.961776722Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-10T18:19:08.189 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.962239715Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-10T18:19:08.189 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.962580642Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-10T18:19:08.189 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.962966735Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-10T18:19:08.189 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.963294217Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-10T18:19:08.189 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.963837266Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-10T18:19:08.189 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.964403023Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-10T18:19:08.189 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.964716863Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-10T18:19:08.189 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.965291895Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-10T18:19:08.189 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.965850857Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-10T18:19:08.190 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.966274027Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-10T18:19:08.190 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.966643111Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-10T18:19:08.190 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.967075446Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-10T18:19:08.190 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.967517483Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-10T18:19:08.190 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.967950097Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-10T18:19:08.190 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.968378691Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-10T18:19:08.190 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.968676947Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-10T18:19:08.190 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.968972483Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-10T18:19:08.190 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.969574496Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-10T18:19:08.190 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.970162709Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-10T18:19:08.190 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.970517156Z level=info msg="Executing migration" id="create quota table v1" 2024-09-10T18:19:08.190 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.970959046Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-10T18:19:08.190 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.971439054Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-10T18:19:08.191 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.97173334Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-10T18:19:08.191 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.97220129Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-10T18:19:08.191 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.972633583Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-10T18:19:08.191 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.973294502Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-10T18:19:08.191 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.973631728Z level=info msg="Executing migration" id="create session table" 2024-09-10T18:19:08.191 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.974080538Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-10T18:19:08.191 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.974437515Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-10T18:19:08.191 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.974734981Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-10T18:19:08.191 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.975164748Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-10T18:19:08.191 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.975593614Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-10T18:19:08.191 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.975870348Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-10T18:19:08.191 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.976192566Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-10T18:19:08.192 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.976483346Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-10T18:19:08.192 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.976793001Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-10T18:19:08.192 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.977256286Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-10T18:19:08.192 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.977567386Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-10T18:19:08.192 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.978187246Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-10T18:19:08.192 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.978551695Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-10T18:19:08.192 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.979206708Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-10T18:19:08.192 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.979829482Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-10T18:19:08.192 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.980172482Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-10T18:19:08.192 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.980647568Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-10T18:19:08.193 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:19:08.194 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.981134904Z level=info msg="Executing migration" id="create alert table v1" 2024-09-10T18:19:08.194 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.981899668Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-10T18:19:08.194 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.982872916Z level=info msg="Executing migration" id="add index alert state" 2024-09-10T18:19:08.194 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.983556563Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-10T18:19:08.194 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.984011199Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-10T18:19:08.194 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.984432932Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-10T18:19:08.194 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.984897252Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-10T18:19:08.194 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.98535055Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-10T18:19:08.194 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.986895336Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-10T18:19:08.194 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.987315582Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-10T18:19:08.195 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.987749661Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-10T18:19:08.195 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.988104147Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-10T18:19:08.195 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.988522547Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-10T18:19:08.195 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.988907513Z level=info msg="Executing migration" id="Add column is_default" 2024-09-10T18:19:08.195 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.989631911Z level=info msg="Executing migration" id="Add column frequency" 2024-09-10T18:19:08.195 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.990327936Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-10T18:19:08.195 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.990984894Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-10T18:19:08.195 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.991692114Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-10T18:19:08.195 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.992138785Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-10T18:19:08.195 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.992472669Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.992755204Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.993211472Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.993682779Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.994094592Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.995019375Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.995957794Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.996749064Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.997640452Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:07.998645307Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.001310846Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.003800349Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.006803974Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.009440269Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-10T18:19:08.196 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.012061417Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-10T18:19:08.197 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.015100335Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-10T18:19:08.197 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.017775835Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-10T18:19:08.197 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.020346997Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-10T18:19:08.197 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.023247385Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-10T18:19:08.197 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.02596308Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-10T18:19:08.197 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.028566887Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-10T18:19:08.197 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.031514812Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-10T18:19:08.197 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.034182925Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-10T18:19:08.197 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.036890358Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-10T18:19:08.197 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.039522663Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-10T18:19:08.197 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.042049037Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-10T18:19:08.197 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.044651478Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-10T18:19:08.197 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.047374482Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.050270925Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.052649428Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.055254292Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.057964145Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.060671338Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.06311107Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.065862776Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.068570696Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.071105285Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.073882053Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.076535813Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.079291825Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.081748999Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-10T18:19:08.198 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.084378208Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-10T18:19:08.199 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.086766578Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-10T18:19:08.199 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.089461303Z level=info msg="Executing migration" id="Move region to single row" 2024-09-10T18:19:08.199 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.092074363Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-10T18:19:08.199 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.092837821Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-10T18:19:08.199 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.093640512Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-10T18:19:08.199 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.094614003Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-10T18:19:08.199 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.095336081Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-10T18:19:08.199 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.096243665Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-10T18:19:08.199 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.09700393Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-10T18:19:08.199 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.097903949Z level=info msg="Executing migration" id="create test_data table" 2024-09-10T18:19:08.199 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.098585167Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-10T18:19:08.199 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.099006731Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-10T18:19:08.199 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.099885316Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-10T18:19:08.200 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.100804895Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-10T18:19:08.200 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.101211771Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-10T18:19:08.200 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.102173205Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-10T18:19:08.200 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.102886664Z level=info msg="Executing migration" id="create team table" 2024-09-10T18:19:08.200 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.103892634Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-10T18:19:08.200 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.104613496Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-10T18:19:08.200 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.105465804Z level=info msg="Executing migration" id="create team member table" 2024-09-10T18:19:08.200 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.106264419Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-10T18:19:08.200 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.107108379Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-10T18:19:08.200 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.107969923Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-10T18:19:08.200 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.108902867Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-10T18:19:08.200 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.109682593Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-10T18:19:08.200 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.110547814Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-10T18:19:08.201 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.111464397Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-10T18:19:08.201 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.112266564Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-10T18:19:08.201 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.113132363Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-10T18:19:08.201 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.113959421Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-10T18:19:08.201 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.114967621Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-10T18:19:08.201 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.115697313Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-10T18:19:08.201 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.11671363Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-10T18:19:08.201 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.117170733Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-10T18:19:08.201 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.118055042Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-10T18:19:08.201 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.11890263Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-10T18:19:08.201 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.119774042Z level=info msg="Executing migration" id="create tag table" 2024-09-10T18:19:08.201 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.120171373Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-10T18:19:08.201 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.121142649Z level=info msg="Executing migration" id="create login attempt table" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.121867446Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.124703748Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.125131319Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.127998785Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.130770838Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.13365451Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.133968494Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.136863884Z level=info msg="Executing migration" id="create user auth table" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.139595934Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.142445991Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.142765656Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.145611936Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.148455216Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-10T18:19:08.202 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.151255646Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-10T18:19:08.203 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.152040765Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-10T18:19:08.203 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.154936904Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-10T18:19:08.203 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.157668896Z level=info msg="Executing migration" id="create server_lock table" 2024-09-10T18:19:08.203 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.160540325Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-10T18:19:08.203 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.163110292Z level=info msg="Executing migration" id="create user auth token table" 2024-09-10T18:19:08.203 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.165898668Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-10T18:19:08.203 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.168607805Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-10T18:19:08.203 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.171233309Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-10T18:19:08.203 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.17175845Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-10T18:19:08.203 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.17460296Z level=info msg="Executing migration" id="create cache_data table" 2024-09-10T18:19:08.203 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.177291223Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-10T18:19:08.203 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.180003249Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-10T18:19:08.203 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.180642151Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-10T18:19:08.204 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.183548633Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-10T18:19:08.204 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.186075991Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-10T18:19:08.204 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.18880887Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-10T18:19:08.204 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.189319392Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-10T18:19:08.443 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.192006067Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-10T18:19:08.443 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.194749878Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-10T18:19:08.443 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.197411063Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-10T18:19:08.444 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.197937039Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-10T18:19:08.444 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.200903452Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-10T18:19:08.444 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.201625456Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-10T18:19:08.444 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.20260041Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-10T18:19:08.444 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.203595744Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-10T18:19:08.444 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.204549455Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-10T18:19:08.444 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.205296506Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-10T18:19:08.444 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.206079528Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-10T18:19:08.444 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.206514361Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-10T18:19:08.444 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.207308176Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-10T18:19:08.444 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.20763166Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-10T18:19:08.444 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.208186354Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-10T18:19:08.445 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.209025693Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-10T18:19:08.445 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.209469068Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-10T18:19:08.445 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.210344246Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-10T18:19:08.445 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.211188711Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-10T18:19:08.445 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.212036543Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-10T18:19:08.445 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.212473449Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-10T18:19:08.445 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.215852326Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-10T18:19:08.445 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.219206262Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-10T18:19:08.445 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.220054367Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-10T18:19:08.445 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.220867419Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-10T18:19:08.446 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.221715404Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-10T18:19:08.446 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.222559651Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-10T18:19:08.446 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.2233881Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-10T18:19:08.446 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.223856788Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-10T18:19:08.446 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.224804371Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-10T18:19:08.446 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.225528087Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-10T18:19:08.446 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.226379756Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-10T18:19:08.446 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.227260007Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-10T18:19:08.446 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.228112649Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-10T18:19:08.446 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.229037817Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-10T18:19:08.446 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.229832435Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-10T18:19:08.446 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.230665869Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-10T18:19:08.446 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.231504912Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-10T18:19:08.447 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.23248483Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-10T18:19:08.447 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.23352326Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-10T18:19:08.447 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.234372981Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-10T18:19:08.447 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.237340565Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-10T18:19:08.447 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.240137259Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-10T18:19:08.447 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.242799819Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-10T18:19:08.447 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.24327846Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-10T18:19:08.447 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.246248545Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-10T18:19:08.447 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.248940958Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-10T18:19:08.447 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.251485874Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-10T18:19:08.447 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.252386206Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-10T18:19:08.447 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.255262628Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-10T18:19:08.447 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.258031899Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-10T18:19:08.448 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.260895632Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-10T18:19:08.448 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.261299841Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-10T18:19:08.452 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.26419663Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-10T18:19:08.452 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.267147937Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-10T18:19:08.452 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.269790632Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-10T18:19:08.452 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.270286717Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-10T18:19:08.452 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.272860019Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-10T18:19:08.453 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.275681777Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-10T18:19:08.453 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.278325058Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-10T18:19:08.453 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.279225288Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-10T18:19:08.453 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.282061971Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-10T18:19:08.453 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.284762279Z level=info msg="Executing migration" id="create alert_image table" 2024-09-10T18:19:08.453 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.28748182Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-10T18:19:08.453 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.288009165Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-10T18:19:08.453 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.29071324Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-10T18:19:08.453 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.293406536Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-10T18:19:08.454 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.29610115Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:19:08.454 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.296335208Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:19:08.454 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.296617094Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-10T18:19:08.454 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.29948896Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-10T18:19:08.454 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.300085542Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-10T18:19:08.454 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.300209554Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-10T18:19:08.454 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.300326101Z level=info msg="alerts found to migrate" alerts=0 2024-09-10T18:19:08.454 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.302158604Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-10T18:19:08.454 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.305013708Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-10T18:19:08.454 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.30545166Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-10T18:19:08.454 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.306292058Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-10T18:19:08.454 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.307190846Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-10T18:19:08.454 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.307854968Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-10T18:19:08.455 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.3081492Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-10T18:19:08.455 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.309108808Z level=info msg="Executing migration" id="create data_keys table" 2024-09-10T18:19:08.455 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.30982084Z level=info msg="Executing migration" id="create secrets table" 2024-09-10T18:19:08.455 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.310760718Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-10T18:19:08.455 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.314978862Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-10T18:19:08.455 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.315877326Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-10T18:19:08.455 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.316719172Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-10T18:19:08.455 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.320950414Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-10T18:19:08.455 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.325303985Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-10T18:19:08.455 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.326079045Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-10T18:19:08.455 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.327026118Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-10T18:19:08.455 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.32748965Z level=info msg="Executing migration" id="create permission table" 2024-09-10T18:19:08.455 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.327968043Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-10T18:19:08.456 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.328733751Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-10T18:19:08.456 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.329723187Z level=info msg="Executing migration" id="create role table" 2024-09-10T18:19:08.456 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.330459966Z level=info msg="Executing migration" id="add column display_name" 2024-09-10T18:19:08.456 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.331413044Z level=info msg="Executing migration" id="add column group_name" 2024-09-10T18:19:08.456 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.332317981Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-10T18:19:08.456 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.333098054Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-10T18:19:08.456 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.334105414Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-10T18:19:08.456 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.334766702Z level=info msg="Executing migration" id="create team role table" 2024-09-10T18:19:08.456 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.335685977Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-10T18:19:08.456 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.336391286Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-10T18:19:08.456 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.337379283Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-10T18:19:08.456 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.337935258Z level=info msg="Executing migration" id="create user role table" 2024-09-10T18:19:08.456 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.34061458Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-10T18:19:08.457 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.343148206Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-10T18:19:08.457 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.346031168Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-10T18:19:08.457 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.346459002Z level=info msg="Executing migration" id="create builtin role table" 2024-09-10T18:19:08.457 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.349060286Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-10T18:19:08.457 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.351964696Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-10T18:19:08.457 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.35471797Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-10T18:19:08.457 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.355721555Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-10T18:19:08.457 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.358727297Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-10T18:19:08.457 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.361307801Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-10T18:19:08.457 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.361779166Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-10T18:19:08.457 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.362210723Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-10T18:19:08.457 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.362578287Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-10T18:19:08.457 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.363001854Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-10T18:19:08.458 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.363971042Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-10T18:19:08.458 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.364362687Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-10T18:19:08.458 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.364776969Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-10T18:19:08.458 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.365067017Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-10T18:19:08.458 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.36546507Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-10T18:19:08.458 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.365880698Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-10T18:19:08.458 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.366279436Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-10T18:19:08.458 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.366552217Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-10T18:19:08.459 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.366888287Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-10T18:19:08.459 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.367293174Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-10T18:19:08.459 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.367717556Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-10T18:19:08.459 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.368676474Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-10T18:19:08.459 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.368974783Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-10T18:19:08.459 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.369399436Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-10T18:19:08.459 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.369807505Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-10T18:19:08.459 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.370222572Z level=info msg="Executing migration" id="add correlation config column" 2024-09-10T18:19:08.459 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.371195207Z level=info msg="Executing migration" id="create entity_events table" 2024-09-10T18:19:08.459 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.37160304Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-10T18:19:08.459 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.374198337Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:19:08.459 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.374294573Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:19:08.459 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.374572215Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:19:08.460 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.374645676Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:19:08.460 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.374884744Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-10T18:19:08.460 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.375288225Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-10T18:19:08.460 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.377728792Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:19:08.460 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.378196703Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:19:08.460 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.38061552Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:19:08.460 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.383179358Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:19:08.460 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.383749287Z level=info msg="Executing migration" id="Drop public config table" 2024-09-10T18:19:08.460 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.386589809Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-10T18:19:08.460 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.389300608Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:19:08.460 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.391772426Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:19:08.461 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.39223848Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-10T18:19:08.461 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.395145482Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-10T18:19:08.461 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.398201666Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-10T18:19:08.461 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.401029536Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-10T18:19:08.461 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.402204369Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-10T18:19:08.461 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.404764603Z level=info msg="Executing migration" id="add share column" 2024-09-10T18:19:08.461 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.407632383Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-10T18:19:08.461 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.410166802Z level=info msg="Executing migration" id="create file table" 2024-09-10T18:19:08.461 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.410630818Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-10T18:19:08.461 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.413459582Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-10T18:19:08.461 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.415873339Z level=info msg="Executing migration" id="create file_meta table" 2024-09-10T18:19:08.461 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.416537307Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-10T18:19:08.462 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.417040953Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-10T18:19:08.462 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.419660777Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-10T18:19:08.462 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.422389461Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-10T18:19:08.462 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.423135748Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-10T18:19:08.462 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.423591428Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-10T18:19:08.462 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.424640334Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-10T18:19:08.462 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.425485296Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-10T18:19:08.462 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.426273983Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-10T18:19:08.462 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.426710674Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-10T18:19:08.462 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.427487483Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-10T18:19:08.462 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.428381977Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-10T18:19:08.462 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.429643337Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-10T18:19:08.463 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.430692006Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-10T18:19:08.463 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.431469156Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-10T18:19:08.463 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.441528789Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-10T18:19:08.463 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.442346819Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-10T18:19:08.463 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.442845354Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-10T18:19:08.741 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.446282776Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-10T18:19:08.741 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.447047748Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-10T18:19:08.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.447836089Z level=info msg="Executing migration" id="create folder table" 2024-09-10T18:19:08.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.448349696Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-10T18:19:08.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.449211502Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-10T18:19:08.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.450018259Z level=info msg="Executing migration" id="Update folder title length" 2024-09-10T18:19:08.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.450830163Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-10T18:19:08.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=migrator t=2024-09-10T18:19:08.451339672Z level=info msg="migrations completed" performed=485 skipped=0 duration=663.337166ms 2024-09-10T18:19:08.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=sqlstore t=2024-09-10T18:19:08.451836284Z level=info msg="Created default organization" 2024-09-10T18:19:08.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=plugin.loader t=2024-09-10T18:19:08.50721911Z level=info msg="Plugin registered" pluginID=input 2024-09-10T18:19:08.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=plugin.loader t=2024-09-10T18:19:08.535035547Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-10T18:19:08.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=plugin.loader t=2024-09-10T18:19:08.535074166Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-10T18:19:08.743 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=secrets t=2024-09-10T18:19:08.535121244Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-10T18:19:08.743 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=query_data t=2024-09-10T18:19:08.536693941Z level=info msg="Query Service initialization" 2024-09-10T18:19:08.743 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=live.push_http t=2024-09-10T18:19:08.539098248Z level=info msg="Live Push Gateway initialization" 2024-09-10T18:19:09.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:08 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:09.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:08 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:09.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:08 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:09.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:08 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:09.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:08 smithi099 ceph-mon[27037]: Deploying daemon node-exporter.a on smithi099 2024-09-10T18:19:09.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:08 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:09.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:08 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:09.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:08 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:09.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:08 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:09.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:08 smithi099 ceph-mon[37557]: Deploying daemon node-exporter.a on smithi099 2024-09-10T18:19:09.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:09.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:09.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:09.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:09.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:08 smithi133 ceph-mon[31214]: Deploying daemon node-exporter.a on smithi099 2024-09-10T18:19:09.741 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=infra.usagestats.collector t=2024-09-10T18:19:09.25234265Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-10T18:19:09.741 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=provisioning.datasources t=2024-09-10T18:19:09.252669803Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-10T18:19:09.741 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=provisioning.datasources t=2024-09-10T18:19:09.285559429Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-10T18:19:09.741 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=provisioning.datasources t=2024-09-10T18:19:09.31702016Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-10T18:19:09.741 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=provisioning.alerting t=2024-09-10T18:19:09.338249098Z level=info msg="starting to provision alerting" 2024-09-10T18:19:09.741 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=provisioning.alerting t=2024-09-10T18:19:09.338286414Z level=info msg="finished to provision alerting" 2024-09-10T18:19:09.741 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=ngalert.state.manager t=2024-09-10T18:19:09.338370793Z level=info msg="Warming state cache for startup" 2024-09-10T18:19:09.741 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=grafanaStorageLogger t=2024-09-10T18:19:09.338443994Z level=info msg="storage starting" 2024-09-10T18:19:09.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=http.server t=2024-09-10T18:19:09.340302392Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-10T18:19:09.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=sqlstore.transactions t=2024-09-10T18:19:09.349075941Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:19:09.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=ngalert.state.manager t=2024-09-10T18:19:09.349146367Z level=info msg="State cache has been initialized" states=0 duration=10.772535ms 2024-09-10T18:19:09.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=ngalert.multiorg.alertmanager t=2024-09-10T18:19:09.349231265Z level=info msg="starting MultiOrg Alertmanager" 2024-09-10T18:19:09.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=ticker t=2024-09-10T18:19:09.349271967Z level=info msg=starting first_tick=2024-09-10T18:19:10Z 2024-09-10T18:19:09.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=sqlstore.transactions t=2024-09-10T18:19:09.382963563Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:19:09.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=sqlstore.transactions t=2024-09-10T18:19:09.423605637Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:19:09.742 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a[62081]: logger=sqlstore.transactions t=2024-09-10T18:19:09.434872914Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-10T18:19:09.860 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:09 smithi099 ceph-mon[27037]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:09.860 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:09 smithi099 ceph-mon[37557]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:09.986 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:19:09.989 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T18:19:10.229 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:09 smithi133 ceph-mon[31214]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:11.395 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-10T18:19:08.330030+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":927,"num_read_kb":784,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385380,"kb_used_data":6540,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575836,"statfs":{"total":767960285184,"available":765517656064,"internally_reserved":0,"allocated":6696960,"data_stored":3631050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002558"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678517+0000","last_change":"2024-09-10T18:17:47.913353+0000","last_active":"2024-09-10T18:17:58.678517+0000","last_peered":"2024-09-10T18:17:58.678517+0000","last_clean":"2024-09-10T18:17:58.678517+0000","last_became_active":"2024-09-10T18:17:47.913079+0000","last_became_peered":"2024-09-10T18:17:47.913079+0000","last_unstale":"2024-09-10T18:17:58.678517+0000","last_undegraded":"2024-09-10T18:17:58.678517+0000","last_fullsized":"2024-09-10T18:17:58.678517+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.992131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647394+0000","last_change":"2024-09-10T18:17:49.922310+0000","last_active":"2024-09-10T18:17:58.647394+0000","last_peered":"2024-09-10T18:17:58.647394+0000","last_clean":"2024-09-10T18:17:58.647394+0000","last_became_active":"2024-09-10T18:17:49.922005+0000","last_became_peered":"2024-09-10T18:17:49.922005+0000","last_unstale":"2024-09-10T18:17:58.647394+0000","last_undegraded":"2024-09-10T18:17:58.647394+0000","last_fullsized":"2024-09-10T18:17:58.647394+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:39.378712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648558+0000","last_change":"2024-09-10T18:17:51.936100+0000","last_active":"2024-09-10T18:17:58.648558+0000","last_peered":"2024-09-10T18:17:58.648558+0000","last_clean":"2024-09-10T18:17:58.648558+0000","last_became_active":"2024-09-10T18:17:51.935629+0000","last_became_peered":"2024-09-10T18:17:51.935629+0000","last_unstale":"2024-09-10T18:17:58.648558+0000","last_undegraded":"2024-09-10T18:17:58.648558+0000","last_fullsized":"2024-09-10T18:17:58.648558+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:12.856098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647571+0000","last_change":"2024-09-10T18:17:53.947886+0000","last_active":"2024-09-10T18:17:58.647571+0000","last_peered":"2024-09-10T18:17:58.647571+0000","last_clean":"2024-09-10T18:17:58.647571+0000","last_became_active":"2024-09-10T18:17:53.947572+0000","last_became_peered":"2024-09-10T18:17:53.947572+0000","last_unstale":"2024-09-10T18:17:58.647571+0000","last_undegraded":"2024-09-10T18:17:58.647571+0000","last_fullsized":"2024-09-10T18:17:58.647571+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:52.841817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686377+0000","last_change":"2024-09-10T18:17:54.187299+0000","last_active":"2024-09-10T18:17:58.686377+0000","last_peered":"2024-09-10T18:17:58.686377+0000","last_clean":"2024-09-10T18:17:58.686377+0000","last_became_active":"2024-09-10T18:17:54.186708+0000","last_became_peered":"2024-09-10T18:17:54.186708+0000","last_unstale":"2024-09-10T18:17:58.686377+0000","last_undegraded":"2024-09-10T18:17:58.686377+0000","last_fullsized":"2024-09-10T18:17:58.686377+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:41.488807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686876+0000","last_change":"2024-09-10T18:17:47.912779+0000","last_active":"2024-09-10T18:17:58.686876+0000","last_peered":"2024-09-10T18:17:58.686876+0000","last_clean":"2024-09-10T18:17:58.686876+0000","last_became_active":"2024-09-10T18:17:47.912369+0000","last_became_peered":"2024-09-10T18:17:47.912369+0000","last_unstale":"2024-09-10T18:17:58.686876+0000","last_undegraded":"2024-09-10T18:17:58.686876+0000","last_fullsized":"2024-09-10T18:17:58.686876+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:19.832774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686384+0000","last_change":"2024-09-10T18:17:49.930063+0000","last_active":"2024-09-10T18:17:58.686384+0000","last_peered":"2024-09-10T18:17:58.686384+0000","last_clean":"2024-09-10T18:17:58.686384+0000","last_became_active":"2024-09-10T18:17:49.929379+0000","last_became_peered":"2024-09-10T18:17:49.929379+0000","last_unstale":"2024-09-10T18:17:58.686384+0000","last_undegraded":"2024-09-10T18:17:58.686384+0000","last_fullsized":"2024-09-10T18:17:58.686384+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:24.111351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647374+0000","last_change":"2024-09-10T18:17:51.946448+0000","last_active":"2024-09-10T18:17:58.647374+0000","last_peered":"2024-09-10T18:17:58.647374+0000","last_clean":"2024-09-10T18:17:58.647374+0000","last_became_active":"2024-09-10T18:17:51.946170+0000","last_became_peered":"2024-09-10T18:17:51.946170+0000","last_unstale":"2024-09-10T18:17:58.647374+0000","last_undegraded":"2024-09-10T18:17:58.647374+0000","last_fullsized":"2024-09-10T18:17:58.647374+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:04.898014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961659+0000","last_change":"2024-09-10T18:17:47.914996+0000","last_active":"2024-09-10T18:17:58.961659+0000","last_peered":"2024-09-10T18:17:58.961659+0000","last_clean":"2024-09-10T18:17:58.961659+0000","last_became_active":"2024-09-10T18:17:47.914722+0000","last_became_peered":"2024-09-10T18:17:47.914722+0000","last_unstale":"2024-09-10T18:17:58.961659+0000","last_undegraded":"2024-09-10T18:17:58.961659+0000","last_fullsized":"2024-09-10T18:17:58.961659+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:24.340748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647745+0000","last_change":"2024-09-10T18:17:49.926098+0000","last_active":"2024-09-10T18:17:58.647745+0000","last_peered":"2024-09-10T18:17:58.647745+0000","last_clean":"2024-09-10T18:17:58.647745+0000","last_became_active":"2024-09-10T18:17:49.925857+0000","last_became_peered":"2024-09-10T18:17:49.925857+0000","last_unstale":"2024-09-10T18:17:58.647745+0000","last_undegraded":"2024-09-10T18:17:58.647745+0000","last_fullsized":"2024-09-10T18:17:58.647745+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:25.710523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961398+0000","last_change":"2024-09-10T18:17:51.941125+0000","last_active":"2024-09-10T18:17:58.961398+0000","last_peered":"2024-09-10T18:17:58.961398+0000","last_clean":"2024-09-10T18:17:58.961398+0000","last_became_active":"2024-09-10T18:17:51.940227+0000","last_became_peered":"2024-09-10T18:17:51.940227+0000","last_unstale":"2024-09-10T18:17:58.961398+0000","last_undegraded":"2024-09-10T18:17:58.961398+0000","last_fullsized":"2024-09-10T18:17:58.961398+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:25.218335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.677945+0000","last_change":"2024-09-10T18:17:53.955325+0000","last_active":"2024-09-10T18:17:58.677945+0000","last_peered":"2024-09-10T18:17:58.677945+0000","last_clean":"2024-09-10T18:17:58.677945+0000","last_became_active":"2024-09-10T18:17:53.955034+0000","last_became_peered":"2024-09-10T18:17:53.955034+0000","last_unstale":"2024-09-10T18:17:58.677945+0000","last_undegraded":"2024-09-10T18:17:58.677945+0000","last_fullsized":"2024-09-10T18:17:58.677945+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:49.821426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961764+0000","last_change":"2024-09-10T18:17:47.920214+0000","last_active":"2024-09-10T18:17:58.961764+0000","last_peered":"2024-09-10T18:17:58.961764+0000","last_clean":"2024-09-10T18:17:58.961764+0000","last_became_active":"2024-09-10T18:17:47.919890+0000","last_became_peered":"2024-09-10T18:17:47.919890+0000","last_unstale":"2024-09-10T18:17:58.961764+0000","last_undegraded":"2024-09-10T18:17:58.961764+0000","last_fullsized":"2024-09-10T18:17:58.961764+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:13.428652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647760+0000","last_change":"2024-09-10T18:17:49.927753+0000","last_active":"2024-09-10T18:17:58.647760+0000","last_peered":"2024-09-10T18:17:58.647760+0000","last_clean":"2024-09-10T18:17:58.647760+0000","last_became_active":"2024-09-10T18:17:49.927476+0000","last_became_peered":"2024-09-10T18:17:49.927476+0000","last_unstale":"2024-09-10T18:17:58.647760+0000","last_undegraded":"2024-09-10T18:17:58.647760+0000","last_fullsized":"2024-09-10T18:17:58.647760+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:24.357068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961488+0000","last_change":"2024-09-10T18:17:51.940403+0000","last_active":"2024-09-10T18:17:58.961488+0000","last_peered":"2024-09-10T18:17:58.961488+0000","last_clean":"2024-09-10T18:17:58.961488+0000","last_became_active":"2024-09-10T18:17:51.939298+0000","last_became_peered":"2024-09-10T18:17:51.939298+0000","last_unstale":"2024-09-10T18:17:58.961488+0000","last_undegraded":"2024-09-10T18:17:58.961488+0000","last_fullsized":"2024-09-10T18:17:58.961488+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:58.780971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961190+0000","last_change":"2024-09-10T18:17:53.949036+0000","last_active":"2024-09-10T18:17:58.961190+0000","last_peered":"2024-09-10T18:17:58.961190+0000","last_clean":"2024-09-10T18:17:58.961190+0000","last_became_active":"2024-09-10T18:17:53.948754+0000","last_became_peered":"2024-09-10T18:17:53.948754+0000","last_unstale":"2024-09-10T18:17:58.961190+0000","last_undegraded":"2024-09-10T18:17:58.961190+0000","last_fullsized":"2024-09-10T18:17:58.961190+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:41.493158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648190+0000","last_change":"2024-09-10T18:17:54.187718+0000","last_active":"2024-09-10T18:17:58.648190+0000","last_peered":"2024-09-10T18:17:58.648190+0000","last_clean":"2024-09-10T18:17:58.648190+0000","last_became_active":"2024-09-10T18:17:54.187436+0000","last_became_peered":"2024-09-10T18:17:54.187436+0000","last_unstale":"2024-09-10T18:17:58.648190+0000","last_undegraded":"2024-09-10T18:17:58.648190+0000","last_fullsized":"2024-09-10T18:17:58.648190+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:05.281186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"52'1","reported_seq":27,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678613+0000","last_change":"2024-09-10T18:17:47.912942+0000","last_active":"2024-09-10T18:17:58.678613+0000","last_peered":"2024-09-10T18:17:58.678613+0000","last_clean":"2024-09-10T18:17:58.678613+0000","last_became_active":"2024-09-10T18:17:47.912638+0000","last_became_peered":"2024-09-10T18:17:47.912638+0000","last_unstale":"2024-09-10T18:17:58.678613+0000","last_undegraded":"2024-09-10T18:17:58.678613+0000","last_fullsized":"2024-09-10T18:17:58.678613+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:12.335422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.649040+0000","last_change":"2024-09-10T18:17:49.929932+0000","last_active":"2024-09-10T18:17:58.649040+0000","last_peered":"2024-09-10T18:17:58.649040+0000","last_clean":"2024-09-10T18:17:58.649040+0000","last_became_active":"2024-09-10T18:17:49.929643+0000","last_became_peered":"2024-09-10T18:17:49.929643+0000","last_unstale":"2024-09-10T18:17:58.649040+0000","last_undegraded":"2024-09-10T18:17:58.649040+0000","last_fullsized":"2024-09-10T18:17:58.649040+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:56.577829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648075+0000","last_change":"2024-09-10T18:17:51.940485+0000","last_active":"2024-09-10T18:17:58.648075+0000","last_peered":"2024-09-10T18:17:58.648075+0000","last_clean":"2024-09-10T18:17:58.648075+0000","last_became_active":"2024-09-10T18:17:51.940168+0000","last_became_peered":"2024-09-10T18:17:51.940168+0000","last_unstale":"2024-09-10T18:17:58.648075+0000","last_undegraded":"2024-09-10T18:17:58.648075+0000","last_fullsized":"2024-09-10T18:17:58.648075+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:48.045412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686873+0000","last_change":"2024-09-10T18:17:54.186437+0000","last_active":"2024-09-10T18:17:58.686873+0000","last_peered":"2024-09-10T18:17:58.686873+0000","last_clean":"2024-09-10T18:17:58.686873+0000","last_became_active":"2024-09-10T18:17:54.186098+0000","last_became_peered":"2024-09-10T18:17:54.186098+0000","last_unstale":"2024-09-10T18:17:58.686873+0000","last_undegraded":"2024-09-10T18:17:58.686873+0000","last_fullsized":"2024-09-10T18:17:58.686873+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:48:09.715096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647812+0000","last_change":"2024-09-10T18:17:47.917385+0000","last_active":"2024-09-10T18:17:58.647812+0000","last_peered":"2024-09-10T18:17:58.647812+0000","last_clean":"2024-09-10T18:17:58.647812+0000","last_became_active":"2024-09-10T18:17:47.917131+0000","last_became_peered":"2024-09-10T18:17:47.917131+0000","last_unstale":"2024-09-10T18:17:58.647812+0000","last_undegraded":"2024-09-10T18:17:58.647812+0000","last_fullsized":"2024-09-10T18:17:58.647812+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:19.619168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686659+0000","last_change":"2024-09-10T18:17:49.928588+0000","last_active":"2024-09-10T18:17:58.686659+0000","last_peered":"2024-09-10T18:17:58.686659+0000","last_clean":"2024-09-10T18:17:58.686659+0000","last_became_active":"2024-09-10T18:17:49.928310+0000","last_became_peered":"2024-09-10T18:17:49.928310+0000","last_unstale":"2024-09-10T18:17:58.686659+0000","last_undegraded":"2024-09-10T18:17:58.686659+0000","last_fullsized":"2024-09-10T18:17:58.686659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:12.617190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647893+0000","last_change":"2024-09-10T18:17:51.947269+0000","last_active":"2024-09-10T18:17:58.647893+0000","last_peered":"2024-09-10T18:17:58.647893+0000","last_clean":"2024-09-10T18:17:58.647893+0000","last_became_active":"2024-09-10T18:17:51.946866+0000","last_became_peered":"2024-09-10T18:17:51.946866+0000","last_unstale":"2024-09-10T18:17:58.647893+0000","last_undegraded":"2024-09-10T18:17:58.647893+0000","last_fullsized":"2024-09-10T18:17:58.647893+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:49.464097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962136+0000","last_change":"2024-09-10T18:17:53.948845+0000","last_active":"2024-09-10T18:17:58.962136+0000","last_peered":"2024-09-10T18:17:58.962136+0000","last_clean":"2024-09-10T18:17:58.962136+0000","last_became_active":"2024-09-10T18:17:53.948570+0000","last_became_peered":"2024-09-10T18:17:53.948570+0000","last_unstale":"2024-09-10T18:17:58.962136+0000","last_undegraded":"2024-09-10T18:17:58.962136+0000","last_fullsized":"2024-09-10T18:17:58.962136+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:51.344212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648065+0000","last_change":"2024-09-10T18:17:47.919574+0000","last_active":"2024-09-10T18:17:58.648065+0000","last_peered":"2024-09-10T18:17:58.648065+0000","last_clean":"2024-09-10T18:17:58.648065+0000","last_became_active":"2024-09-10T18:17:47.919207+0000","last_became_peered":"2024-09-10T18:17:47.919207+0000","last_unstale":"2024-09-10T18:17:58.648065+0000","last_undegraded":"2024-09-10T18:17:58.648065+0000","last_fullsized":"2024-09-10T18:17:58.648065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:52.343060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678478+0000","last_change":"2024-09-10T18:17:50.099083+0000","last_active":"2024-09-10T18:17:58.678478+0000","last_peered":"2024-09-10T18:17:58.678478+0000","last_clean":"2024-09-10T18:17:58.678478+0000","last_became_active":"2024-09-10T18:17:50.098823+0000","last_became_peered":"2024-09-10T18:17:50.098823+0000","last_unstale":"2024-09-10T18:17:58.678478+0000","last_undegraded":"2024-09-10T18:17:58.678478+0000","last_fullsized":"2024-09-10T18:17:58.678478+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:16.154758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107613+0000","last_change":"2024-09-10T18:17:51.945246+0000","last_active":"2024-09-10T18:19:01.107613+0000","last_peered":"2024-09-10T18:19:01.107613+0000","last_clean":"2024-09-10T18:19:01.107613+0000","last_became_active":"2024-09-10T18:17:51.943976+0000","last_became_peered":"2024-09-10T18:17:51.943976+0000","last_unstale":"2024-09-10T18:19:01.107613+0000","last_undegraded":"2024-09-10T18:19:01.107613+0000","last_fullsized":"2024-09-10T18:19:01.107613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:51.322968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648790+0000","last_change":"2024-09-10T18:17:49.929859+0000","last_active":"2024-09-10T18:17:58.648790+0000","last_peered":"2024-09-10T18:17:58.648790+0000","last_clean":"2024-09-10T18:17:58.648790+0000","last_became_active":"2024-09-10T18:17:49.929299+0000","last_became_peered":"2024-09-10T18:17:49.929299+0000","last_unstale":"2024-09-10T18:17:58.648790+0000","last_undegraded":"2024-09-10T18:17:58.648790+0000","last_fullsized":"2024-09-10T18:17:58.648790+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:02.276340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686278+0000","last_change":"2024-09-10T18:17:47.911501+0000","last_active":"2024-09-10T18:17:58.686278+0000","last_peered":"2024-09-10T18:17:58.686278+0000","last_clean":"2024-09-10T18:17:58.686278+0000","last_became_active":"2024-09-10T18:17:47.911224+0000","last_became_peered":"2024-09-10T18:17:47.911224+0000","last_unstale":"2024-09-10T18:17:58.686278+0000","last_undegraded":"2024-09-10T18:17:58.686278+0000","last_fullsized":"2024-09-10T18:17:58.686278+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:17.441640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107613+0000","last_change":"2024-09-10T18:17:51.943508+0000","last_active":"2024-09-10T18:19:01.107613+0000","last_peered":"2024-09-10T18:19:01.107613+0000","last_clean":"2024-09-10T18:19:01.107613+0000","last_became_active":"2024-09-10T18:17:51.943208+0000","last_became_peered":"2024-09-10T18:17:51.943208+0000","last_unstale":"2024-09-10T18:19:01.107613+0000","last_undegraded":"2024-09-10T18:19:01.107613+0000","last_fullsized":"2024-09-10T18:19:01.107613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:28.575117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961278+0000","last_change":"2024-09-10T18:17:53.955381+0000","last_active":"2024-09-10T18:17:58.961278+0000","last_peered":"2024-09-10T18:17:58.961278+0000","last_clean":"2024-09-10T18:17:58.961278+0000","last_became_active":"2024-09-10T18:17:53.955112+0000","last_became_peered":"2024-09-10T18:17:53.955112+0000","last_unstale":"2024-09-10T18:17:58.961278+0000","last_undegraded":"2024-09-10T18:17:58.961278+0000","last_fullsized":"2024-09-10T18:17:58.961278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:56.887383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686091+0000","last_change":"2024-09-10T18:17:50.098970+0000","last_active":"2024-09-10T18:17:58.686091+0000","last_peered":"2024-09-10T18:17:58.686091+0000","last_clean":"2024-09-10T18:17:58.686091+0000","last_became_active":"2024-09-10T18:17:50.098718+0000","last_became_peered":"2024-09-10T18:17:50.098718+0000","last_unstale":"2024-09-10T18:17:58.686091+0000","last_undegraded":"2024-09-10T18:17:58.686091+0000","last_fullsized":"2024-09-10T18:17:58.686091+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:17.432478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.687005+0000","last_change":"2024-09-10T18:17:47.912791+0000","last_active":"2024-09-10T18:17:58.687005+0000","last_peered":"2024-09-10T18:17:58.687005+0000","last_clean":"2024-09-10T18:17:58.687005+0000","last_became_active":"2024-09-10T18:17:47.912537+0000","last_became_peered":"2024-09-10T18:17:47.912537+0000","last_unstale":"2024-09-10T18:17:58.687005+0000","last_undegraded":"2024-09-10T18:17:58.687005+0000","last_fullsized":"2024-09-10T18:17:58.687005+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:28.199019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647423+0000","last_change":"2024-09-10T18:17:51.948141+0000","last_active":"2024-09-10T18:17:58.647423+0000","last_peered":"2024-09-10T18:17:58.647423+0000","last_clean":"2024-09-10T18:17:58.647423+0000","last_became_active":"2024-09-10T18:17:51.947662+0000","last_became_peered":"2024-09-10T18:17:51.947662+0000","last_unstale":"2024-09-10T18:17:58.647423+0000","last_undegraded":"2024-09-10T18:17:58.647423+0000","last_fullsized":"2024-09-10T18:17:58.647423+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:54.239287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647721+0000","last_change":"2024-09-10T18:17:53.947259+0000","last_active":"2024-09-10T18:17:58.647721+0000","last_peered":"2024-09-10T18:17:58.647721+0000","last_clean":"2024-09-10T18:17:58.647721+0000","last_became_active":"2024-09-10T18:17:53.946967+0000","last_became_peered":"2024-09-10T18:17:53.946967+0000","last_unstale":"2024-09-10T18:17:58.647721+0000","last_undegraded":"2024-09-10T18:17:58.647721+0000","last_fullsized":"2024-09-10T18:17:58.647721+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:00.131810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647497+0000","last_change":"2024-09-10T18:17:49.920000+0000","last_active":"2024-09-10T18:17:58.647497+0000","last_peered":"2024-09-10T18:17:58.647497+0000","last_clean":"2024-09-10T18:17:58.647497+0000","last_became_active":"2024-09-10T18:17:49.919667+0000","last_became_peered":"2024-09-10T18:17:49.919667+0000","last_unstale":"2024-09-10T18:17:58.647497+0000","last_undegraded":"2024-09-10T18:17:58.647497+0000","last_fullsized":"2024-09-10T18:17:58.647497+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:09.676833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678607+0000","last_change":"2024-09-10T18:17:47.907334+0000","last_active":"2024-09-10T18:17:58.678607+0000","last_peered":"2024-09-10T18:17:58.678607+0000","last_clean":"2024-09-10T18:17:58.678607+0000","last_became_active":"2024-09-10T18:17:47.907070+0000","last_became_peered":"2024-09-10T18:17:47.907070+0000","last_unstale":"2024-09-10T18:17:58.678607+0000","last_undegraded":"2024-09-10T18:17:58.678607+0000","last_fullsized":"2024-09-10T18:17:58.678607+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:14.976361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107815+0000","last_change":"2024-09-10T18:17:51.945298+0000","last_active":"2024-09-10T18:19:01.107815+0000","last_peered":"2024-09-10T18:19:01.107815+0000","last_clean":"2024-09-10T18:19:01.107815+0000","last_became_active":"2024-09-10T18:17:51.944841+0000","last_became_peered":"2024-09-10T18:17:51.944841+0000","last_unstale":"2024-09-10T18:19:01.107815+0000","last_undegraded":"2024-09-10T18:19:01.107815+0000","last_fullsized":"2024-09-10T18:19:01.107815+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:41.514771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961715+0000","last_change":"2024-09-10T18:17:54.190656+0000","last_active":"2024-09-10T18:17:58.961715+0000","last_peered":"2024-09-10T18:17:58.961715+0000","last_clean":"2024-09-10T18:17:58.961715+0000","last_became_active":"2024-09-10T18:17:54.189698+0000","last_became_peered":"2024-09-10T18:17:54.189698+0000","last_unstale":"2024-09-10T18:17:58.961715+0000","last_undegraded":"2024-09-10T18:17:58.961715+0000","last_fullsized":"2024-09-10T18:17:58.961715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:40.652122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648965+0000","last_change":"2024-09-10T18:17:49.927162+0000","last_active":"2024-09-10T18:17:58.648965+0000","last_peered":"2024-09-10T18:17:58.648965+0000","last_clean":"2024-09-10T18:17:58.648965+0000","last_became_active":"2024-09-10T18:17:49.926851+0000","last_became_peered":"2024-09-10T18:17:49.926851+0000","last_unstale":"2024-09-10T18:17:58.648965+0000","last_undegraded":"2024-09-10T18:17:58.648965+0000","last_fullsized":"2024-09-10T18:17:58.648965+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:48.113651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961727+0000","last_change":"2024-09-10T18:17:47.913460+0000","last_active":"2024-09-10T18:17:58.961727+0000","last_peered":"2024-09-10T18:17:58.961727+0000","last_clean":"2024-09-10T18:17:58.961727+0000","last_became_active":"2024-09-10T18:17:47.913189+0000","last_became_peered":"2024-09-10T18:17:47.913189+0000","last_unstale":"2024-09-10T18:17:58.961727+0000","last_undegraded":"2024-09-10T18:17:58.961727+0000","last_fullsized":"2024-09-10T18:17:58.961727+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:22.970274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107719+0000","last_change":"2024-09-10T18:17:51.939209+0000","last_active":"2024-09-10T18:19:01.107719+0000","last_peered":"2024-09-10T18:19:01.107719+0000","last_clean":"2024-09-10T18:19:01.107719+0000","last_became_active":"2024-09-10T18:17:51.938931+0000","last_became_peered":"2024-09-10T18:17:51.938931+0000","last_unstale":"2024-09-10T18:19:01.107719+0000","last_undegraded":"2024-09-10T18:19:01.107719+0000","last_fullsized":"2024-09-10T18:19:01.107719+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:44.974616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678120+0000","last_change":"2024-09-10T18:17:53.953867+0000","last_active":"2024-09-10T18:17:58.678120+0000","last_peered":"2024-09-10T18:17:58.678120+0000","last_clean":"2024-09-10T18:17:58.678120+0000","last_became_active":"2024-09-10T18:17:53.953069+0000","last_became_peered":"2024-09-10T18:17:53.953069+0000","last_unstale":"2024-09-10T18:17:58.678120+0000","last_undegraded":"2024-09-10T18:17:58.678120+0000","last_fullsized":"2024-09-10T18:17:58.678120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:31.926307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678431+0000","last_change":"2024-09-10T18:17:50.099431+0000","last_active":"2024-09-10T18:17:58.678431+0000","last_peered":"2024-09-10T18:17:58.678431+0000","last_clean":"2024-09-10T18:17:58.678431+0000","last_became_active":"2024-09-10T18:17:50.099142+0000","last_became_peered":"2024-09-10T18:17:50.099142+0000","last_unstale":"2024-09-10T18:17:58.678431+0000","last_undegraded":"2024-09-10T18:17:58.678431+0000","last_fullsized":"2024-09-10T18:17:58.678431+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:58.443405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648761+0000","last_change":"2024-09-10T18:17:47.915490+0000","last_active":"2024-09-10T18:17:58.648761+0000","last_peered":"2024-09-10T18:17:58.648761+0000","last_clean":"2024-09-10T18:17:58.648761+0000","last_became_active":"2024-09-10T18:17:47.915011+0000","last_became_peered":"2024-09-10T18:17:47.915011+0000","last_unstale":"2024-09-10T18:17:58.648761+0000","last_undegraded":"2024-09-10T18:17:58.648761+0000","last_fullsized":"2024-09-10T18:17:58.648761+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:41.633692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962131+0000","last_change":"2024-09-10T18:17:51.944137+0000","last_active":"2024-09-10T18:17:58.962131+0000","last_peered":"2024-09-10T18:17:58.962131+0000","last_clean":"2024-09-10T18:17:58.962131+0000","last_became_active":"2024-09-10T18:17:51.943793+0000","last_became_peered":"2024-09-10T18:17:51.943793+0000","last_unstale":"2024-09-10T18:17:58.962131+0000","last_undegraded":"2024-09-10T18:17:58.962131+0000","last_fullsized":"2024-09-10T18:17:58.962131+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:30.728743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647456+0000","last_change":"2024-09-10T18:17:54.186331+0000","last_active":"2024-09-10T18:17:58.647456+0000","last_peered":"2024-09-10T18:17:58.647456+0000","last_clean":"2024-09-10T18:17:58.647456+0000","last_became_active":"2024-09-10T18:17:54.185947+0000","last_became_peered":"2024-09-10T18:17:54.185947+0000","last_unstale":"2024-09-10T18:17:58.647456+0000","last_undegraded":"2024-09-10T18:17:58.647456+0000","last_fullsized":"2024-09-10T18:17:58.647456+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:18.108739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107719+0000","last_change":"2024-09-10T18:17:49.931497+0000","last_active":"2024-09-10T18:19:01.107719+0000","last_peered":"2024-09-10T18:19:01.107719+0000","last_clean":"2024-09-10T18:19:01.107719+0000","last_became_active":"2024-09-10T18:17:49.931193+0000","last_became_peered":"2024-09-10T18:17:49.931193+0000","last_unstale":"2024-09-10T18:19:01.107719+0000","last_undegraded":"2024-09-10T18:19:01.107719+0000","last_fullsized":"2024-09-10T18:19:01.107719+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:18.393114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647977+0000","last_change":"2024-09-10T18:17:47.914843+0000","last_active":"2024-09-10T18:17:58.647977+0000","last_peered":"2024-09-10T18:17:58.647977+0000","last_clean":"2024-09-10T18:17:58.647977+0000","last_became_active":"2024-09-10T18:17:47.914569+0000","last_became_peered":"2024-09-10T18:17:47.914569+0000","last_unstale":"2024-09-10T18:17:58.647977+0000","last_undegraded":"2024-09-10T18:17:58.647977+0000","last_fullsized":"2024-09-10T18:17:58.647977+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:19.792761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961840+0000","last_change":"2024-09-10T18:17:51.933437+0000","last_active":"2024-09-10T18:17:58.961840+0000","last_peered":"2024-09-10T18:17:58.961840+0000","last_clean":"2024-09-10T18:17:58.961840+0000","last_became_active":"2024-09-10T18:17:47.915524+0000","last_became_peered":"2024-09-10T18:17:47.915524+0000","last_unstale":"2024-09-10T18:17:58.961840+0000","last_undegraded":"2024-09-10T18:17:58.961840+0000","last_fullsized":"2024-09-10T18:17:58.961840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:40:32.994132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043679999999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678401+0000","last_change":"2024-09-10T18:17:51.943377+0000","last_active":"2024-09-10T18:17:58.678401+0000","last_peered":"2024-09-10T18:17:58.678401+0000","last_clean":"2024-09-10T18:17:58.678401+0000","last_became_active":"2024-09-10T18:17:51.942963+0000","last_became_peered":"2024-09-10T18:17:51.942963+0000","last_unstale":"2024-09-10T18:17:58.678401+0000","last_undegraded":"2024-09-10T18:17:58.678401+0000","last_fullsized":"2024-09-10T18:17:58.678401+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:02.069882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686700+0000","last_change":"2024-09-10T18:17:53.945868+0000","last_active":"2024-09-10T18:17:58.686700+0000","last_peered":"2024-09-10T18:17:58.686700+0000","last_clean":"2024-09-10T18:17:58.686700+0000","last_became_active":"2024-09-10T18:17:53.945559+0000","last_became_peered":"2024-09-10T18:17:53.945559+0000","last_unstale":"2024-09-10T18:17:58.686700+0000","last_undegraded":"2024-09-10T18:17:58.686700+0000","last_fullsized":"2024-09-10T18:17:58.686700+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:14.679190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648684+0000","last_change":"2024-09-10T18:17:49.923094+0000","last_active":"2024-09-10T18:17:58.648684+0000","last_peered":"2024-09-10T18:17:58.648684+0000","last_clean":"2024-09-10T18:17:58.648684+0000","last_became_active":"2024-09-10T18:17:49.922256+0000","last_became_peered":"2024-09-10T18:17:49.922256+0000","last_unstale":"2024-09-10T18:17:58.648684+0000","last_undegraded":"2024-09-10T18:17:58.648684+0000","last_fullsized":"2024-09-10T18:17:58.648684+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:53.639177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647663+0000","last_change":"2024-09-10T18:17:47.915334+0000","last_active":"2024-09-10T18:17:58.647663+0000","last_peered":"2024-09-10T18:17:58.647663+0000","last_clean":"2024-09-10T18:17:58.647663+0000","last_became_active":"2024-09-10T18:17:47.915087+0000","last_became_peered":"2024-09-10T18:17:47.915087+0000","last_unstale":"2024-09-10T18:17:58.647663+0000","last_undegraded":"2024-09-10T18:17:58.647663+0000","last_fullsized":"2024-09-10T18:17:58.647663+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.868408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648126+0000","last_change":"2024-09-10T18:17:51.930420+0000","last_active":"2024-09-10T18:17:58.648126+0000","last_peered":"2024-09-10T18:17:58.648126+0000","last_clean":"2024-09-10T18:17:58.648126+0000","last_became_active":"2024-09-10T18:17:47.908038+0000","last_became_peered":"2024-09-10T18:17:47.908038+0000","last_unstale":"2024-09-10T18:17:58.648126+0000","last_undegraded":"2024-09-10T18:17:58.648126+0000","last_fullsized":"2024-09-10T18:17:58.648126+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:43.957190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011789140000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648672+0000","last_change":"2024-09-10T18:17:51.939553+0000","last_active":"2024-09-10T18:17:58.648672+0000","last_peered":"2024-09-10T18:17:58.648672+0000","last_clean":"2024-09-10T18:17:58.648672+0000","last_became_active":"2024-09-10T18:17:51.939291+0000","last_became_peered":"2024-09-10T18:17:51.939291+0000","last_unstale":"2024-09-10T18:17:58.648672+0000","last_undegraded":"2024-09-10T18:17:58.648672+0000","last_fullsized":"2024-09-10T18:17:58.648672+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:54.887490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962883+0000","last_change":"2024-09-10T18:17:54.189214+0000","last_active":"2024-09-10T18:17:58.962883+0000","last_peered":"2024-09-10T18:17:58.962883+0000","last_clean":"2024-09-10T18:17:58.962883+0000","last_became_active":"2024-09-10T18:17:54.188939+0000","last_became_peered":"2024-09-10T18:17:54.188939+0000","last_unstale":"2024-09-10T18:17:58.962883+0000","last_undegraded":"2024-09-10T18:17:58.962883+0000","last_fullsized":"2024-09-10T18:17:58.962883+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:44.556501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678534+0000","last_change":"2024-09-10T18:17:49.923899+0000","last_active":"2024-09-10T18:17:58.678534+0000","last_peered":"2024-09-10T18:17:58.678534+0000","last_clean":"2024-09-10T18:17:58.678534+0000","last_became_active":"2024-09-10T18:17:49.923569+0000","last_became_peered":"2024-09-10T18:17:49.923569+0000","last_unstale":"2024-09-10T18:17:58.678534+0000","last_undegraded":"2024-09-10T18:17:58.678534+0000","last_fullsized":"2024-09-10T18:17:58.678534+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:53.078822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678003+0000","last_change":"2024-09-10T18:17:47.909157+0000","last_active":"2024-09-10T18:17:58.678003+0000","last_peered":"2024-09-10T18:17:58.678003+0000","last_clean":"2024-09-10T18:17:58.678003+0000","last_became_active":"2024-09-10T18:17:47.908868+0000","last_became_peered":"2024-09-10T18:17:47.908868+0000","last_unstale":"2024-09-10T18:17:58.678003+0000","last_undegraded":"2024-09-10T18:17:58.678003+0000","last_fullsized":"2024-09-10T18:17:58.678003+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:10.715648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"60'5","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:07.002311+0000","last_change":"2024-09-10T18:17:51.929853+0000","last_active":"2024-09-10T18:19:07.002311+0000","last_peered":"2024-09-10T18:19:07.002311+0000","last_clean":"2024-09-10T18:19:07.002311+0000","last_became_active":"2024-09-10T18:17:47.918342+0000","last_became_peered":"2024-09-10T18:17:47.918342+0000","last_unstale":"2024-09-10T18:19:07.002311+0000","last_undegraded":"2024-09-10T18:19:07.002311+0000","last_fullsized":"2024-09-10T18:19:07.002311+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:34.520922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035361600000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962287+0000","last_change":"2024-09-10T18:17:51.943234+0000","last_active":"2024-09-10T18:17:58.962287+0000","last_peered":"2024-09-10T18:17:58.962287+0000","last_clean":"2024-09-10T18:17:58.962287+0000","last_became_active":"2024-09-10T18:17:51.941422+0000","last_became_peered":"2024-09-10T18:17:51.941422+0000","last_unstale":"2024-09-10T18:17:58.962287+0000","last_undegraded":"2024-09-10T18:17:58.962287+0000","last_fullsized":"2024-09-10T18:17:58.962287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:40.453088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648326+0000","last_change":"2024-09-10T18:17:53.953464+0000","last_active":"2024-09-10T18:17:58.648326+0000","last_peered":"2024-09-10T18:17:58.648326+0000","last_clean":"2024-09-10T18:17:58.648326+0000","last_became_active":"2024-09-10T18:17:53.953064+0000","last_became_peered":"2024-09-10T18:17:53.953064+0000","last_unstale":"2024-09-10T18:17:58.648326+0000","last_undegraded":"2024-09-10T18:17:58.648326+0000","last_fullsized":"2024-09-10T18:17:58.648326+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:59.398334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961861+0000","last_change":"2024-09-10T18:17:50.099216+0000","last_active":"2024-09-10T18:17:58.961861+0000","last_peered":"2024-09-10T18:17:58.961861+0000","last_clean":"2024-09-10T18:17:58.961861+0000","last_became_active":"2024-09-10T18:17:50.098878+0000","last_became_peered":"2024-09-10T18:17:50.098878+0000","last_unstale":"2024-09-10T18:17:58.961861+0000","last_undegraded":"2024-09-10T18:17:58.961861+0000","last_fullsized":"2024-09-10T18:17:58.961861+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:11.706968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686495+0000","last_change":"2024-09-10T18:17:47.912779+0000","last_active":"2024-09-10T18:17:58.686495+0000","last_peered":"2024-09-10T18:17:58.686495+0000","last_clean":"2024-09-10T18:17:58.686495+0000","last_became_active":"2024-09-10T18:17:47.912271+0000","last_became_peered":"2024-09-10T18:17:47.912271+0000","last_unstale":"2024-09-10T18:17:58.686495+0000","last_undegraded":"2024-09-10T18:17:58.686495+0000","last_fullsized":"2024-09-10T18:17:58.686495+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:56.789693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:18:24.365476+0000","last_change":"2024-09-10T18:17:33.855354+0000","last_active":"2024-09-10T18:18:24.365476+0000","last_peered":"2024-09-10T18:18:24.365476+0000","last_clean":"2024-09-10T18:18:24.365476+0000","last_became_active":"2024-09-10T18:17:33.849783+0000","last_became_peered":"2024-09-10T18:17:33.849783+0000","last_unstale":"2024-09-10T18:18:24.365476+0000","last_undegraded":"2024-09-10T18:18:24.365476+0000","last_fullsized":"2024-09-10T18:18:24.365476+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:15:42.260515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:15:42.260515+0000","last_clean_scrub_stamp":"2024-09-10T18:15:42.260515+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:57.467633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962261+0000","last_change":"2024-09-10T18:17:51.939177+0000","last_active":"2024-09-10T18:17:58.962261+0000","last_peered":"2024-09-10T18:17:58.962261+0000","last_clean":"2024-09-10T18:17:58.962261+0000","last_became_active":"2024-09-10T18:17:51.938805+0000","last_became_peered":"2024-09-10T18:17:51.938805+0000","last_unstale":"2024-09-10T18:17:58.962261+0000","last_undegraded":"2024-09-10T18:17:58.962261+0000","last_fullsized":"2024-09-10T18:17:58.962261+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:41.738130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962102+0000","last_change":"2024-09-10T18:17:53.948344+0000","last_active":"2024-09-10T18:17:58.962102+0000","last_peered":"2024-09-10T18:17:58.962102+0000","last_clean":"2024-09-10T18:17:58.962102+0000","last_became_active":"2024-09-10T18:17:53.947663+0000","last_became_peered":"2024-09-10T18:17:53.947663+0000","last_unstale":"2024-09-10T18:17:58.962102+0000","last_undegraded":"2024-09-10T18:17:58.962102+0000","last_fullsized":"2024-09-10T18:17:58.962102+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:08.629389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648272+0000","last_change":"2024-09-10T18:17:49.926517+0000","last_active":"2024-09-10T18:17:58.648272+0000","last_peered":"2024-09-10T18:17:58.648272+0000","last_clean":"2024-09-10T18:17:58.648272+0000","last_became_active":"2024-09-10T18:17:49.926265+0000","last_became_peered":"2024-09-10T18:17:49.926265+0000","last_unstale":"2024-09-10T18:17:58.648272+0000","last_undegraded":"2024-09-10T18:17:58.648272+0000","last_fullsized":"2024-09-10T18:17:58.648272+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:08.568044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647587+0000","last_change":"2024-09-10T18:17:47.912442+0000","last_active":"2024-09-10T18:17:58.647587+0000","last_peered":"2024-09-10T18:17:58.647587+0000","last_clean":"2024-09-10T18:17:58.647587+0000","last_became_active":"2024-09-10T18:17:47.912125+0000","last_became_peered":"2024-09-10T18:17:47.912125+0000","last_unstale":"2024-09-10T18:17:58.647587+0000","last_undegraded":"2024-09-10T18:17:58.647587+0000","last_fullsized":"2024-09-10T18:17:58.647587+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:44.304839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962094+0000","last_change":"2024-09-10T18:17:51.941024+0000","last_active":"2024-09-10T18:17:58.962094+0000","last_peered":"2024-09-10T18:17:58.962094+0000","last_clean":"2024-09-10T18:17:58.962094+0000","last_became_active":"2024-09-10T18:17:51.940713+0000","last_became_peered":"2024-09-10T18:17:51.940713+0000","last_unstale":"2024-09-10T18:17:58.962094+0000","last_undegraded":"2024-09-10T18:17:58.962094+0000","last_fullsized":"2024-09-10T18:17:58.962094+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:20.919047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686451+0000","last_change":"2024-09-10T18:17:54.186836+0000","last_active":"2024-09-10T18:17:58.686451+0000","last_peered":"2024-09-10T18:17:58.686451+0000","last_clean":"2024-09-10T18:17:58.686451+0000","last_became_active":"2024-09-10T18:17:54.186071+0000","last_became_peered":"2024-09-10T18:17:54.186071+0000","last_unstale":"2024-09-10T18:17:58.686451+0000","last_undegraded":"2024-09-10T18:17:58.686451+0000","last_fullsized":"2024-09-10T18:17:58.686451+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:50.730457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648161+0000","last_change":"2024-09-10T18:17:49.922410+0000","last_active":"2024-09-10T18:17:58.648161+0000","last_peered":"2024-09-10T18:17:58.648161+0000","last_clean":"2024-09-10T18:17:58.648161+0000","last_became_active":"2024-09-10T18:17:49.921884+0000","last_became_peered":"2024-09-10T18:17:49.921884+0000","last_unstale":"2024-09-10T18:17:58.648161+0000","last_undegraded":"2024-09-10T18:17:58.648161+0000","last_fullsized":"2024-09-10T18:17:58.648161+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:42.616685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962337+0000","last_change":"2024-09-10T18:17:47.915001+0000","last_active":"2024-09-10T18:17:58.962337+0000","last_peered":"2024-09-10T18:17:58.962337+0000","last_clean":"2024-09-10T18:17:58.962337+0000","last_became_active":"2024-09-10T18:17:47.914721+0000","last_became_peered":"2024-09-10T18:17:47.914721+0000","last_unstale":"2024-09-10T18:17:58.962337+0000","last_undegraded":"2024-09-10T18:17:58.962337+0000","last_fullsized":"2024-09-10T18:17:58.962337+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:18.364951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648402+0000","last_change":"2024-09-10T18:17:51.940619+0000","last_active":"2024-09-10T18:17:58.648402+0000","last_peered":"2024-09-10T18:17:58.648402+0000","last_clean":"2024-09-10T18:17:58.648402+0000","last_became_active":"2024-09-10T18:17:51.940359+0000","last_became_peered":"2024-09-10T18:17:51.940359+0000","last_unstale":"2024-09-10T18:17:58.648402+0000","last_undegraded":"2024-09-10T18:17:58.648402+0000","last_fullsized":"2024-09-10T18:17:58.648402+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:39.062953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648418+0000","last_change":"2024-09-10T18:17:53.952413+0000","last_active":"2024-09-10T18:17:58.648418+0000","last_peered":"2024-09-10T18:17:58.648418+0000","last_clean":"2024-09-10T18:17:58.648418+0000","last_became_active":"2024-09-10T18:17:53.951759+0000","last_became_peered":"2024-09-10T18:17:53.951759+0000","last_unstale":"2024-09-10T18:17:58.648418+0000","last_undegraded":"2024-09-10T18:17:58.648418+0000","last_fullsized":"2024-09-10T18:17:58.648418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:43.378170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647909+0000","last_change":"2024-09-10T18:17:49.927821+0000","last_active":"2024-09-10T18:17:58.647909+0000","last_peered":"2024-09-10T18:17:58.647909+0000","last_clean":"2024-09-10T18:17:58.647909+0000","last_became_active":"2024-09-10T18:17:49.927525+0000","last_became_peered":"2024-09-10T18:17:49.927525+0000","last_unstale":"2024-09-10T18:17:58.647909+0000","last_undegraded":"2024-09-10T18:17:58.647909+0000","last_fullsized":"2024-09-10T18:17:58.647909+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:49.385183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686592+0000","last_change":"2024-09-10T18:17:47.913415+0000","last_active":"2024-09-10T18:17:58.686592+0000","last_peered":"2024-09-10T18:17:58.686592+0000","last_clean":"2024-09-10T18:17:58.686592+0000","last_became_active":"2024-09-10T18:17:47.913157+0000","last_became_peered":"2024-09-10T18:17:47.913157+0000","last_unstale":"2024-09-10T18:17:58.686592+0000","last_undegraded":"2024-09-10T18:17:58.686592+0000","last_fullsized":"2024-09-10T18:17:58.686592+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:15.019106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107731+0000","last_change":"2024-09-10T18:17:51.939593+0000","last_active":"2024-09-10T18:19:01.107731+0000","last_peered":"2024-09-10T18:19:01.107731+0000","last_clean":"2024-09-10T18:19:01.107731+0000","last_became_active":"2024-09-10T18:17:51.939244+0000","last_became_peered":"2024-09-10T18:17:51.939244+0000","last_unstale":"2024-09-10T18:19:01.107731+0000","last_undegraded":"2024-09-10T18:19:01.107731+0000","last_fullsized":"2024-09-10T18:19:01.107731+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:15.630619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648106+0000","last_change":"2024-09-10T18:17:53.947479+0000","last_active":"2024-09-10T18:17:58.648106+0000","last_peered":"2024-09-10T18:17:58.648106+0000","last_clean":"2024-09-10T18:17:58.648106+0000","last_became_active":"2024-09-10T18:17:53.947093+0000","last_became_peered":"2024-09-10T18:17:53.947093+0000","last_unstale":"2024-09-10T18:17:58.648106+0000","last_undegraded":"2024-09-10T18:17:58.648106+0000","last_fullsized":"2024-09-10T18:17:58.648106+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:23.472424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678409+0000","last_change":"2024-09-10T18:17:49.922842+0000","last_active":"2024-09-10T18:17:58.678409+0000","last_peered":"2024-09-10T18:17:58.678409+0000","last_clean":"2024-09-10T18:17:58.678409+0000","last_became_active":"2024-09-10T18:17:49.922246+0000","last_became_peered":"2024-09-10T18:17:49.922246+0000","last_unstale":"2024-09-10T18:17:58.678409+0000","last_undegraded":"2024-09-10T18:17:58.678409+0000","last_fullsized":"2024-09-10T18:17:58.678409+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:22.929374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961887+0000","last_change":"2024-09-10T18:17:47.912560+0000","last_active":"2024-09-10T18:17:58.961887+0000","last_peered":"2024-09-10T18:17:58.961887+0000","last_clean":"2024-09-10T18:17:58.961887+0000","last_became_active":"2024-09-10T18:17:47.912269+0000","last_became_peered":"2024-09-10T18:17:47.912269+0000","last_unstale":"2024-09-10T18:17:58.961887+0000","last_undegraded":"2024-09-10T18:17:58.961887+0000","last_fullsized":"2024-09-10T18:17:58.961887+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:28.461851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647954+0000","last_change":"2024-09-10T18:17:51.940710+0000","last_active":"2024-09-10T18:17:58.647954+0000","last_peered":"2024-09-10T18:17:58.647954+0000","last_clean":"2024-09-10T18:17:58.647954+0000","last_became_active":"2024-09-10T18:17:51.940421+0000","last_became_peered":"2024-09-10T18:17:51.940421+0000","last_unstale":"2024-09-10T18:17:58.647954+0000","last_undegraded":"2024-09-10T18:17:58.647954+0000","last_fullsized":"2024-09-10T18:17:58.647954+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:29.833364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678016+0000","last_change":"2024-09-10T18:17:53.953547+0000","last_active":"2024-09-10T18:17:58.678016+0000","last_peered":"2024-09-10T18:17:58.678016+0000","last_clean":"2024-09-10T18:17:58.678016+0000","last_became_active":"2024-09-10T18:17:53.952997+0000","last_became_peered":"2024-09-10T18:17:53.952997+0000","last_unstale":"2024-09-10T18:17:58.678016+0000","last_undegraded":"2024-09-10T18:17:58.678016+0000","last_fullsized":"2024-09-10T18:17:58.678016+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:08.141228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962236+0000","last_change":"2024-09-10T18:17:50.099303+0000","last_active":"2024-09-10T18:17:58.962236+0000","last_peered":"2024-09-10T18:17:58.962236+0000","last_clean":"2024-09-10T18:17:58.962236+0000","last_became_active":"2024-09-10T18:17:50.098970+0000","last_became_peered":"2024-09-10T18:17:50.098970+0000","last_unstale":"2024-09-10T18:17:58.962236+0000","last_undegraded":"2024-09-10T18:17:58.962236+0000","last_fullsized":"2024-09-10T18:17:58.962236+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:59.689308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.963108+0000","last_change":"2024-09-10T18:17:47.916889+0000","last_active":"2024-09-10T18:17:58.963108+0000","last_peered":"2024-09-10T18:17:58.963108+0000","last_clean":"2024-09-10T18:17:58.963108+0000","last_became_active":"2024-09-10T18:17:47.915651+0000","last_became_peered":"2024-09-10T18:17:47.915651+0000","last_unstale":"2024-09-10T18:17:58.963108+0000","last_undegraded":"2024-09-10T18:17:58.963108+0000","last_fullsized":"2024-09-10T18:17:58.963108+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:50.007519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647573+0000","last_change":"2024-09-10T18:17:51.938964+0000","last_active":"2024-09-10T18:17:58.647573+0000","last_peered":"2024-09-10T18:17:58.647573+0000","last_clean":"2024-09-10T18:17:58.647573+0000","last_became_active":"2024-09-10T18:17:51.938610+0000","last_became_peered":"2024-09-10T18:17:51.938610+0000","last_unstale":"2024-09-10T18:17:58.647573+0000","last_undegraded":"2024-09-10T18:17:58.647573+0000","last_fullsized":"2024-09-10T18:17:58.647573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:33.700812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962832+0000","last_change":"2024-09-10T18:17:53.948301+0000","last_active":"2024-09-10T18:17:58.962832+0000","last_peered":"2024-09-10T18:17:58.962832+0000","last_clean":"2024-09-10T18:17:58.962832+0000","last_became_active":"2024-09-10T18:17:53.947535+0000","last_became_peered":"2024-09-10T18:17:53.947535+0000","last_unstale":"2024-09-10T18:17:58.962832+0000","last_undegraded":"2024-09-10T18:17:58.962832+0000","last_fullsized":"2024-09-10T18:17:58.962832+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:57.956421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647457+0000","last_change":"2024-09-10T18:17:49.927734+0000","last_active":"2024-09-10T18:17:58.647457+0000","last_peered":"2024-09-10T18:17:58.647457+0000","last_clean":"2024-09-10T18:17:58.647457+0000","last_became_active":"2024-09-10T18:17:49.927418+0000","last_became_peered":"2024-09-10T18:17:49.927418+0000","last_unstale":"2024-09-10T18:17:58.647457+0000","last_undegraded":"2024-09-10T18:17:58.647457+0000","last_fullsized":"2024-09-10T18:17:58.647457+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:22.287667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.963100+0000","last_change":"2024-09-10T18:17:47.917389+0000","last_active":"2024-09-10T18:17:58.963100+0000","last_peered":"2024-09-10T18:17:58.963100+0000","last_clean":"2024-09-10T18:17:58.963100+0000","last_became_active":"2024-09-10T18:17:47.917108+0000","last_became_peered":"2024-09-10T18:17:47.917108+0000","last_unstale":"2024-09-10T18:17:58.963100+0000","last_undegraded":"2024-09-10T18:17:58.963100+0000","last_fullsized":"2024-09-10T18:17:58.963100+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:38.381474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647613+0000","last_change":"2024-09-10T18:17:51.937905+0000","last_active":"2024-09-10T18:17:58.647613+0000","last_peered":"2024-09-10T18:17:58.647613+0000","last_clean":"2024-09-10T18:17:58.647613+0000","last_became_active":"2024-09-10T18:17:51.937565+0000","last_became_peered":"2024-09-10T18:17:51.937565+0000","last_unstale":"2024-09-10T18:17:58.647613+0000","last_undegraded":"2024-09-10T18:17:58.647613+0000","last_fullsized":"2024-09-10T18:17:58.647613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:49.203573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686372+0000","last_change":"2024-09-10T18:17:53.947136+0000","last_active":"2024-09-10T18:17:58.686372+0000","last_peered":"2024-09-10T18:17:58.686372+0000","last_clean":"2024-09-10T18:17:58.686372+0000","last_became_active":"2024-09-10T18:17:53.946747+0000","last_became_peered":"2024-09-10T18:17:53.946747+0000","last_unstale":"2024-09-10T18:17:58.686372+0000","last_undegraded":"2024-09-10T18:17:58.686372+0000","last_fullsized":"2024-09-10T18:17:58.686372+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:18.722442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961136+0000","last_change":"2024-09-10T18:17:50.100732+0000","last_active":"2024-09-10T18:17:58.961136+0000","last_peered":"2024-09-10T18:17:58.961136+0000","last_clean":"2024-09-10T18:17:58.961136+0000","last_became_active":"2024-09-10T18:17:50.099740+0000","last_became_peered":"2024-09-10T18:17:50.099740+0000","last_unstale":"2024-09-10T18:17:58.961136+0000","last_undegraded":"2024-09-10T18:17:58.961136+0000","last_fullsized":"2024-09-10T18:17:58.961136+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:11.588736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"52'2","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962675+0000","last_change":"2024-09-10T18:17:47.914310+0000","last_active":"2024-09-10T18:17:58.962675+0000","last_peered":"2024-09-10T18:17:58.962675+0000","last_clean":"2024-09-10T18:17:58.962675+0000","last_became_active":"2024-09-10T18:17:47.914072+0000","last_became_peered":"2024-09-10T18:17:47.914072+0000","last_unstale":"2024-09-10T18:17:58.962675+0000","last_undegraded":"2024-09-10T18:17:58.962675+0000","last_fullsized":"2024-09-10T18:17:58.962675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:00.665604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107790+0000","last_change":"2024-09-10T18:17:51.943130+0000","last_active":"2024-09-10T18:19:01.107790+0000","last_peered":"2024-09-10T18:19:01.107790+0000","last_clean":"2024-09-10T18:19:01.107790+0000","last_became_active":"2024-09-10T18:17:51.942774+0000","last_became_peered":"2024-09-10T18:17:51.942774+0000","last_unstale":"2024-09-10T18:19:01.107790+0000","last_undegraded":"2024-09-10T18:19:01.107790+0000","last_fullsized":"2024-09-10T18:19:01.107790+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:16.246552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961478+0000","last_change":"2024-09-10T18:17:54.186495+0000","last_active":"2024-09-10T18:17:58.961478+0000","last_peered":"2024-09-10T18:17:58.961478+0000","last_clean":"2024-09-10T18:17:58.961478+0000","last_became_active":"2024-09-10T18:17:54.186225+0000","last_became_peered":"2024-09-10T18:17:54.186225+0000","last_unstale":"2024-09-10T18:17:58.961478+0000","last_undegraded":"2024-09-10T18:17:58.961478+0000","last_fullsized":"2024-09-10T18:17:58.961478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:42.174818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962001+0000","last_change":"2024-09-10T18:17:47.917528+0000","last_active":"2024-09-10T18:17:58.962001+0000","last_peered":"2024-09-10T18:17:58.962001+0000","last_clean":"2024-09-10T18:17:58.962001+0000","last_became_active":"2024-09-10T18:17:47.917270+0000","last_became_peered":"2024-09-10T18:17:47.917270+0000","last_unstale":"2024-09-10T18:17:58.962001+0000","last_undegraded":"2024-09-10T18:17:58.962001+0000","last_fullsized":"2024-09-10T18:17:58.962001+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:41.867811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686953+0000","last_change":"2024-09-10T18:17:50.099156+0000","last_active":"2024-09-10T18:17:58.686953+0000","last_peered":"2024-09-10T18:17:58.686953+0000","last_clean":"2024-09-10T18:17:58.686953+0000","last_became_active":"2024-09-10T18:17:50.098873+0000","last_became_peered":"2024-09-10T18:17:50.098873+0000","last_unstale":"2024-09-10T18:17:58.686953+0000","last_undegraded":"2024-09-10T18:17:58.686953+0000","last_fullsized":"2024-09-10T18:17:58.686953+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:20.247117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962134+0000","last_change":"2024-09-10T18:17:51.942265+0000","last_active":"2024-09-10T18:17:58.962134+0000","last_peered":"2024-09-10T18:17:58.962134+0000","last_clean":"2024-09-10T18:17:58.962134+0000","last_became_active":"2024-09-10T18:17:51.941957+0000","last_became_peered":"2024-09-10T18:17:51.941957+0000","last_unstale":"2024-09-10T18:17:58.962134+0000","last_undegraded":"2024-09-10T18:17:58.962134+0000","last_fullsized":"2024-09-10T18:17:58.962134+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:49.780993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962896+0000","last_change":"2024-09-10T18:17:54.189989+0000","last_active":"2024-09-10T18:17:58.962896+0000","last_peered":"2024-09-10T18:17:58.962896+0000","last_clean":"2024-09-10T18:17:58.962896+0000","last_became_active":"2024-09-10T18:17:54.189643+0000","last_became_peered":"2024-09-10T18:17:54.189643+0000","last_unstale":"2024-09-10T18:17:58.962896+0000","last_undegraded":"2024-09-10T18:17:58.962896+0000","last_fullsized":"2024-09-10T18:17:58.962896+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:54.564734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678561+0000","last_change":"2024-09-10T18:17:50.100249+0000","last_active":"2024-09-10T18:17:58.678561+0000","last_peered":"2024-09-10T18:17:58.678561+0000","last_clean":"2024-09-10T18:17:58.678561+0000","last_became_active":"2024-09-10T18:17:50.099961+0000","last_became_peered":"2024-09-10T18:17:50.099961+0000","last_unstale":"2024-09-10T18:17:58.678561+0000","last_undegraded":"2024-09-10T18:17:58.678561+0000","last_fullsized":"2024-09-10T18:17:58.678561+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:09.454909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962766+0000","last_change":"2024-09-10T18:17:47.915387+0000","last_active":"2024-09-10T18:17:58.962766+0000","last_peered":"2024-09-10T18:17:58.962766+0000","last_clean":"2024-09-10T18:17:58.962766+0000","last_became_active":"2024-09-10T18:17:47.915143+0000","last_became_peered":"2024-09-10T18:17:47.915143+0000","last_unstale":"2024-09-10T18:17:58.962766+0000","last_undegraded":"2024-09-10T18:17:58.962766+0000","last_fullsized":"2024-09-10T18:17:58.962766+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:47.751991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686891+0000","last_change":"2024-09-10T18:17:51.943838+0000","last_active":"2024-09-10T18:17:58.686891+0000","last_peered":"2024-09-10T18:17:58.686891+0000","last_clean":"2024-09-10T18:17:58.686891+0000","last_became_active":"2024-09-10T18:17:51.943532+0000","last_became_peered":"2024-09-10T18:17:51.943532+0000","last_unstale":"2024-09-10T18:17:58.686891+0000","last_undegraded":"2024-09-10T18:17:58.686891+0000","last_fullsized":"2024-09-10T18:17:58.686891+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:05.292320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648759+0000","last_change":"2024-09-10T18:17:53.951450+0000","last_active":"2024-09-10T18:17:58.648759+0000","last_peered":"2024-09-10T18:17:58.648759+0000","last_clean":"2024-09-10T18:17:58.648759+0000","last_became_active":"2024-09-10T18:17:53.951185+0000","last_became_peered":"2024-09-10T18:17:53.951185+0000","last_unstale":"2024-09-10T18:17:58.648759+0000","last_undegraded":"2024-09-10T18:17:58.648759+0000","last_fullsized":"2024-09-10T18:17:58.648759+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:06.420044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961590+0000","last_change":"2024-09-10T18:17:50.099290+0000","last_active":"2024-09-10T18:17:58.961590+0000","last_peered":"2024-09-10T18:17:58.961590+0000","last_clean":"2024-09-10T18:17:58.961590+0000","last_became_active":"2024-09-10T18:17:50.099015+0000","last_became_peered":"2024-09-10T18:17:50.099015+0000","last_unstale":"2024-09-10T18:17:58.961590+0000","last_undegraded":"2024-09-10T18:17:58.961590+0000","last_fullsized":"2024-09-10T18:17:58.961590+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:15.354709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678185+0000","last_change":"2024-09-10T18:17:47.911957+0000","last_active":"2024-09-10T18:17:58.678185+0000","last_peered":"2024-09-10T18:17:58.678185+0000","last_clean":"2024-09-10T18:17:58.678185+0000","last_became_active":"2024-09-10T18:17:47.911575+0000","last_became_peered":"2024-09-10T18:17:47.911575+0000","last_unstale":"2024-09-10T18:17:58.678185+0000","last_undegraded":"2024-09-10T18:17:58.678185+0000","last_fullsized":"2024-09-10T18:17:58.678185+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:28.065372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107864+0000","last_change":"2024-09-10T18:17:51.942856+0000","last_active":"2024-09-10T18:19:01.107864+0000","last_peered":"2024-09-10T18:19:01.107864+0000","last_clean":"2024-09-10T18:19:01.107864+0000","last_became_active":"2024-09-10T18:17:51.941697+0000","last_became_peered":"2024-09-10T18:17:51.941697+0000","last_unstale":"2024-09-10T18:19:01.107864+0000","last_undegraded":"2024-09-10T18:19:01.107864+0000","last_fullsized":"2024-09-10T18:19:01.107864+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:12.669121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648115+0000","last_change":"2024-09-10T18:17:53.948708+0000","last_active":"2024-09-10T18:17:58.648115+0000","last_peered":"2024-09-10T18:17:58.648115+0000","last_clean":"2024-09-10T18:17:58.648115+0000","last_became_active":"2024-09-10T18:17:53.948404+0000","last_became_peered":"2024-09-10T18:17:53.948404+0000","last_unstale":"2024-09-10T18:17:58.648115+0000","last_undegraded":"2024-09-10T18:17:58.648115+0000","last_fullsized":"2024-09-10T18:17:58.648115+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:14.775697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686909+0000","last_change":"2024-09-10T18:17:50.098867+0000","last_active":"2024-09-10T18:17:58.686909+0000","last_peered":"2024-09-10T18:17:58.686909+0000","last_clean":"2024-09-10T18:17:58.686909+0000","last_became_active":"2024-09-10T18:17:50.098593+0000","last_became_peered":"2024-09-10T18:17:50.098593+0000","last_unstale":"2024-09-10T18:17:58.686909+0000","last_undegraded":"2024-09-10T18:17:58.686909+0000","last_fullsized":"2024-09-10T18:17:58.686909+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:17.229907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961997+0000","last_change":"2024-09-10T18:17:47.913498+0000","last_active":"2024-09-10T18:17:58.961997+0000","last_peered":"2024-09-10T18:17:58.961997+0000","last_clean":"2024-09-10T18:17:58.961997+0000","last_became_active":"2024-09-10T18:17:47.913060+0000","last_became_peered":"2024-09-10T18:17:47.913060+0000","last_unstale":"2024-09-10T18:17:58.961997+0000","last_undegraded":"2024-09-10T18:17:58.961997+0000","last_fullsized":"2024-09-10T18:17:58.961997+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:34:19.558122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107613+0000","last_change":"2024-09-10T18:17:51.941630+0000","last_active":"2024-09-10T18:19:01.107613+0000","last_peered":"2024-09-10T18:19:01.107613+0000","last_clean":"2024-09-10T18:19:01.107613+0000","last_became_active":"2024-09-10T18:17:51.940189+0000","last_became_peered":"2024-09-10T18:17:51.940189+0000","last_unstale":"2024-09-10T18:19:01.107613+0000","last_undegraded":"2024-09-10T18:19:01.107613+0000","last_fullsized":"2024-09-10T18:19:01.107613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:44.342180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678332+0000","last_change":"2024-09-10T18:17:53.956251+0000","last_active":"2024-09-10T18:17:58.678332+0000","last_peered":"2024-09-10T18:17:58.678332+0000","last_clean":"2024-09-10T18:17:58.678332+0000","last_became_active":"2024-09-10T18:17:53.955973+0000","last_became_peered":"2024-09-10T18:17:53.955973+0000","last_unstale":"2024-09-10T18:17:58.678332+0000","last_undegraded":"2024-09-10T18:17:58.678332+0000","last_fullsized":"2024-09-10T18:17:58.678332+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:25.486164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647785+0000","last_change":"2024-09-10T18:17:49.922415+0000","last_active":"2024-09-10T18:17:58.647785+0000","last_peered":"2024-09-10T18:17:58.647785+0000","last_clean":"2024-09-10T18:17:58.647785+0000","last_became_active":"2024-09-10T18:17:49.922127+0000","last_became_peered":"2024-09-10T18:17:49.922127+0000","last_unstale":"2024-09-10T18:17:58.647785+0000","last_undegraded":"2024-09-10T18:17:58.647785+0000","last_fullsized":"2024-09-10T18:17:58.647785+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:18.842298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648033+0000","last_change":"2024-09-10T18:17:47.916076+0000","last_active":"2024-09-10T18:17:58.648033+0000","last_peered":"2024-09-10T18:17:58.648033+0000","last_clean":"2024-09-10T18:17:58.648033+0000","last_became_active":"2024-09-10T18:17:47.915311+0000","last_became_peered":"2024-09-10T18:17:47.915311+0000","last_unstale":"2024-09-10T18:17:58.648033+0000","last_undegraded":"2024-09-10T18:17:58.648033+0000","last_fullsized":"2024-09-10T18:17:58.648033+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:36.286026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648827+0000","last_change":"2024-09-10T18:17:51.938586+0000","last_active":"2024-09-10T18:17:58.648827+0000","last_peered":"2024-09-10T18:17:58.648827+0000","last_clean":"2024-09-10T18:17:58.648827+0000","last_became_active":"2024-09-10T18:17:51.938324+0000","last_became_peered":"2024-09-10T18:17:51.938324+0000","last_unstale":"2024-09-10T18:17:58.648827+0000","last_undegraded":"2024-09-10T18:17:58.648827+0000","last_fullsized":"2024-09-10T18:17:58.648827+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:48.130738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686128+0000","last_change":"2024-09-10T18:17:53.954175+0000","last_active":"2024-09-10T18:17:58.686128+0000","last_peered":"2024-09-10T18:17:58.686128+0000","last_clean":"2024-09-10T18:17:58.686128+0000","last_became_active":"2024-09-10T18:17:53.953796+0000","last_became_peered":"2024-09-10T18:17:53.953796+0000","last_unstale":"2024-09-10T18:17:58.686128+0000","last_undegraded":"2024-09-10T18:17:58.686128+0000","last_fullsized":"2024-09-10T18:17:58.686128+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:46.386483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.649104+0000","last_change":"2024-09-10T18:17:49.929377+0000","last_active":"2024-09-10T18:17:58.649104+0000","last_peered":"2024-09-10T18:17:58.649104+0000","last_clean":"2024-09-10T18:17:58.649104+0000","last_became_active":"2024-09-10T18:17:49.928833+0000","last_became_peered":"2024-09-10T18:17:49.928833+0000","last_unstale":"2024-09-10T18:17:58.649104+0000","last_undegraded":"2024-09-10T18:17:58.649104+0000","last_fullsized":"2024-09-10T18:17:58.649104+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:32.132956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.963048+0000","last_change":"2024-09-10T18:17:47.913760+0000","last_active":"2024-09-10T18:17:58.963048+0000","last_peered":"2024-09-10T18:17:58.963048+0000","last_clean":"2024-09-10T18:17:58.963048+0000","last_became_active":"2024-09-10T18:17:47.913426+0000","last_became_peered":"2024-09-10T18:17:47.913426+0000","last_unstale":"2024-09-10T18:17:58.963048+0000","last_undegraded":"2024-09-10T18:17:58.963048+0000","last_fullsized":"2024-09-10T18:17:58.963048+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:43.518962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686503+0000","last_change":"2024-09-10T18:17:51.938886+0000","last_active":"2024-09-10T18:17:58.686503+0000","last_peered":"2024-09-10T18:17:58.686503+0000","last_clean":"2024-09-10T18:17:58.686503+0000","last_became_active":"2024-09-10T18:17:51.938591+0000","last_became_peered":"2024-09-10T18:17:51.938591+0000","last_unstale":"2024-09-10T18:17:58.686503+0000","last_undegraded":"2024-09-10T18:17:58.686503+0000","last_fullsized":"2024-09-10T18:17:58.686503+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:24.065760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.677845+0000","last_change":"2024-09-10T18:17:53.956943+0000","last_active":"2024-09-10T18:17:58.677845+0000","last_peered":"2024-09-10T18:17:58.677845+0000","last_clean":"2024-09-10T18:17:58.677845+0000","last_became_active":"2024-09-10T18:17:53.956663+0000","last_became_peered":"2024-09-10T18:17:53.956663+0000","last_unstale":"2024-09-10T18:17:58.677845+0000","last_undegraded":"2024-09-10T18:17:58.677845+0000","last_fullsized":"2024-09-10T18:17:58.677845+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:43.143114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686272+0000","last_change":"2024-09-10T18:17:50.099052+0000","last_active":"2024-09-10T18:17:58.686272+0000","last_peered":"2024-09-10T18:17:58.686272+0000","last_clean":"2024-09-10T18:17:58.686272+0000","last_became_active":"2024-09-10T18:17:50.098700+0000","last_became_peered":"2024-09-10T18:17:50.098700+0000","last_unstale":"2024-09-10T18:17:58.686272+0000","last_undegraded":"2024-09-10T18:17:58.686272+0000","last_fullsized":"2024-09-10T18:17:58.686272+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:57.595169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962204+0000","last_change":"2024-09-10T18:17:47.917189+0000","last_active":"2024-09-10T18:17:58.962204+0000","last_peered":"2024-09-10T18:17:58.962204+0000","last_clean":"2024-09-10T18:17:58.962204+0000","last_became_active":"2024-09-10T18:17:47.916930+0000","last_became_peered":"2024-09-10T18:17:47.916930+0000","last_unstale":"2024-09-10T18:17:58.962204+0000","last_undegraded":"2024-09-10T18:17:58.962204+0000","last_fullsized":"2024-09-10T18:17:58.962204+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:09.259941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961072+0000","last_change":"2024-09-10T18:17:51.944246+0000","last_active":"2024-09-10T18:17:58.961072+0000","last_peered":"2024-09-10T18:17:58.961072+0000","last_clean":"2024-09-10T18:17:58.961072+0000","last_became_active":"2024-09-10T18:17:51.943973+0000","last_became_peered":"2024-09-10T18:17:51.943973+0000","last_unstale":"2024-09-10T18:17:58.961072+0000","last_undegraded":"2024-09-10T18:17:58.961072+0000","last_fullsized":"2024-09-10T18:17:58.961072+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:23.600285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686577+0000","last_change":"2024-09-10T18:17:54.186992+0000","last_active":"2024-09-10T18:17:58.686577+0000","last_peered":"2024-09-10T18:17:58.686577+0000","last_clean":"2024-09-10T18:17:58.686577+0000","last_became_active":"2024-09-10T18:17:54.186675+0000","last_became_peered":"2024-09-10T18:17:54.186675+0000","last_unstale":"2024-09-10T18:17:58.686577+0000","last_undegraded":"2024-09-10T18:17:58.686577+0000","last_fullsized":"2024-09-10T18:17:58.686577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:19.450768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686123+0000","last_change":"2024-09-10T18:17:49.929557+0000","last_active":"2024-09-10T18:17:58.686123+0000","last_peered":"2024-09-10T18:17:58.686123+0000","last_clean":"2024-09-10T18:17:58.686123+0000","last_became_active":"2024-09-10T18:17:49.928925+0000","last_became_peered":"2024-09-10T18:17:49.928925+0000","last_unstale":"2024-09-10T18:17:58.686123+0000","last_undegraded":"2024-09-10T18:17:58.686123+0000","last_fullsized":"2024-09-10T18:17:58.686123+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:35.327522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678581+0000","last_change":"2024-09-10T18:17:47.914095+0000","last_active":"2024-09-10T18:17:58.678581+0000","last_peered":"2024-09-10T18:17:58.678581+0000","last_clean":"2024-09-10T18:17:58.678581+0000","last_became_active":"2024-09-10T18:17:47.913545+0000","last_became_peered":"2024-09-10T18:17:47.913545+0000","last_unstale":"2024-09-10T18:17:58.678581+0000","last_undegraded":"2024-09-10T18:17:58.678581+0000","last_fullsized":"2024-09-10T18:17:58.678581+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:56.279468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962303+0000","last_change":"2024-09-10T18:17:51.946475+0000","last_active":"2024-09-10T18:17:58.962303+0000","last_peered":"2024-09-10T18:17:58.962303+0000","last_clean":"2024-09-10T18:17:58.962303+0000","last_became_active":"2024-09-10T18:17:51.946154+0000","last_became_peered":"2024-09-10T18:17:51.946154+0000","last_unstale":"2024-09-10T18:17:58.962303+0000","last_undegraded":"2024-09-10T18:17:58.962303+0000","last_fullsized":"2024-09-10T18:17:58.962303+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:51.519862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961423+0000","last_change":"2024-09-10T18:17:53.948163+0000","last_active":"2024-09-10T18:17:58.961423+0000","last_peered":"2024-09-10T18:17:58.961423+0000","last_clean":"2024-09-10T18:17:58.961423+0000","last_became_active":"2024-09-10T18:17:53.947578+0000","last_became_peered":"2024-09-10T18:17:53.947578+0000","last_unstale":"2024-09-10T18:17:58.961423+0000","last_undegraded":"2024-09-10T18:17:58.961423+0000","last_fullsized":"2024-09-10T18:17:58.961423+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:31.078803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647849+0000","last_change":"2024-09-10T18:17:53.953655+0000","last_active":"2024-09-10T18:17:58.647849+0000","last_peered":"2024-09-10T18:17:58.647849+0000","last_clean":"2024-09-10T18:17:58.647849+0000","last_became_active":"2024-09-10T18:17:53.953374+0000","last_became_peered":"2024-09-10T18:17:53.953374+0000","last_unstale":"2024-09-10T18:17:58.647849+0000","last_undegraded":"2024-09-10T18:17:58.647849+0000","last_fullsized":"2024-09-10T18:17:58.647849+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:44.017375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686219+0000","last_change":"2024-09-10T18:17:47.917657+0000","last_active":"2024-09-10T18:17:58.686219+0000","last_peered":"2024-09-10T18:17:58.686219+0000","last_clean":"2024-09-10T18:17:58.686219+0000","last_became_active":"2024-09-10T18:17:47.917323+0000","last_became_peered":"2024-09-10T18:17:47.917323+0000","last_unstale":"2024-09-10T18:17:58.686219+0000","last_undegraded":"2024-09-10T18:17:58.686219+0000","last_fullsized":"2024-09-10T18:17:58.686219+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:11.321171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961593+0000","last_change":"2024-09-10T18:17:49.925378+0000","last_active":"2024-09-10T18:17:58.961593+0000","last_peered":"2024-09-10T18:17:58.961593+0000","last_clean":"2024-09-10T18:17:58.961593+0000","last_became_active":"2024-09-10T18:17:49.925089+0000","last_became_peered":"2024-09-10T18:17:49.925089+0000","last_unstale":"2024-09-10T18:17:58.961593+0000","last_undegraded":"2024-09-10T18:17:58.961593+0000","last_fullsized":"2024-09-10T18:17:58.961593+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:26.777992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678397+0000","last_change":"2024-09-10T18:17:51.942242+0000","last_active":"2024-09-10T18:17:58.678397+0000","last_peered":"2024-09-10T18:17:58.678397+0000","last_clean":"2024-09-10T18:17:58.678397+0000","last_became_active":"2024-09-10T18:17:51.941974+0000","last_became_peered":"2024-09-10T18:17:51.941974+0000","last_unstale":"2024-09-10T18:17:58.678397+0000","last_undegraded":"2024-09-10T18:17:58.678397+0000","last_fullsized":"2024-09-10T18:17:58.678397+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:13.060700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430229,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298536,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446616,"statfs":{"total":95995035648,"available":95689334784,"internally_reserved":0,"allocated":1212416,"data_stored":823893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.47299999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.48599999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":17.661000000000001,"5min":17.661000000000001,"15min":17.661000000000001},"last":0.46500000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.45800000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.433},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":17.603000000000002,"5min":17.603000000000002,"15min":17.603000000000002},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.41999999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.315},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.35999999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.40400000000000003}]}]},{"osd":6,"up_from":42,"seq":180388626458,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298512,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446640,"statfs":{"total":95995035648,"available":95689359360,"internally_reserved":0,"allocated":1183744,"data_stored":821351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.51500000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.624}]},{"osd":2,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.498},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.63700000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52400000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.373}]},{"osd":5,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.41399999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:18:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.44900000000000001}]}]},{"osd":5,"up_from":36,"seq":154618822687,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":231670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:18:51 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.501,"15min":0.501},"min":{"1min":0.35299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.47499999999999998,"15min":0.47199999999999998},"min":{"1min":0.35699999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Sep 10 18:18:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.34799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.66300000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.5,"15min":0.5},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.69599999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.52000000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:18:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48799999999999999,"15min":0.48899999999999999},"min":{"1min":0.38500000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.58999999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.41999999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:18:51 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.64900000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.39400000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.49099999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:18:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.35499999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.39100000000000001,"15min":0.376},"min":{"1min":0.28199999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.50700000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.437},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.54100000000000004}]},{"osd":7,"last update":"Tue Sep 10 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.56699999999999995}]}]},{"osd":4,"up_from":30,"seq":128849018914,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297956,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447196,"statfs":{"total":95995035648,"available":95689928704,"internally_reserved":0,"allocated":614400,"data_stored":232672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.504,"15min":0.504},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.442}]},{"osd":1,"last update":"Tue Sep 10 18:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.501},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64700000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.52100000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.504,"15min":0.503},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.496,"15min":0.497},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67800000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41499999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67200000000000004,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.47699999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.372,"15min":0.36799999999999999},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.37},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.36599999999999999,"15min":0.35999999999999999},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.35999999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.33300000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:18:34 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.34899999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182439,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297964,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447188,"statfs":{"total":95995035648,"available":95689920512,"internally_reserved":0,"allocated":622592,"data_stored":232130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:19:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.308},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39700000000000002,"15min":0.40799999999999997},"min":{"1min":0.25600000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.442}]},{"osd":1,"last update":"Tue Sep 10 18:19:07 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.64200000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.40799999999999997,"15min":0.41999999999999998},"min":{"1min":0.24299999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.42499999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:19:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.437,"15min":0.44700000000000001},"min":{"1min":0.26600000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.42099999999999999,"15min":0.434},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48999999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.46200000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:18:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68700000000000006,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.371},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.52700000000000002},"min":{"1min":0.375,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:18:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.501},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39800000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":6.5990000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":6.5839999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.47799999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297936,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447216,"statfs":{"total":95995035648,"available":95689949184,"internally_reserved":0,"allocated":593920,"data_stored":232076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35399999999999998,"15min":0.35099999999999998},"min":{"1min":0.22900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44900000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.372,"15min":0.372},"min":{"1min":0.248,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.50800000000000001,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.30299999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.38800000000000001,"15min":0.39300000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40400000000000003,"15min":0.40600000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.313,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.29199999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68000000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.443,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.72199999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.45800000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:18:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.375},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.41699999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.38600000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.432}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":233655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33400000000000002,"15min":0.31900000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.35399999999999998,"15min":0.33800000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.56899999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.22600000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35299999999999998,"15min":0.33900000000000002},"min":{"1min":0.24099999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35499999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.503,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.31}]},{"osd":3,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41099999999999998,"15min":0.41299999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.50800000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41499999999999998,"15min":0.41599999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.27800000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72099999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.39000000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.37}]},{"osd":6,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.57299999999999995}]},{"osd":7,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.378},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.54500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738420,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298540,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446612,"statfs":{"total":95995035648,"available":95689330688,"internally_reserved":0,"allocated":1216512,"data_stored":823603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32400000000000001,"15min":0.30599999999999999},"min":{"1min":0.19400000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.436},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.34899999999999998,"15min":0.32100000000000001},"min":{"1min":0.247,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.36399999999999999,"15min":0.36599999999999999},"min":{"1min":0.219,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.45400000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.39000000000000001,"15min":0.38300000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51600000000000001,"5min":0.753,"15min":0.753},"last":0.41299999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.249,"5min":0.247,"15min":0.247},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.496},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.46200000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:18:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69699999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68500000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Sep 10 18:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.40000000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.67100000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55500000000000005,"15min":0.55900000000000005},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66300000000000003}]},{"osd":6,"last update":"Tue Sep 10 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71399999999999997}]},{"osd":7,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.623}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:19:11.403 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:19:11.661 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:19:11.925 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:11 smithi099 ceph-mon[27037]: from='client.14745 v1:172.21.15.99:0/929751512' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:19:11.926 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:11 smithi099 ceph-mon[27037]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:11.926 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:11 smithi099 ceph-mon[37557]: from='client.14745 v1:172.21.15.99:0/929751512' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:19:11.926 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:11 smithi099 ceph-mon[37557]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:12.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:11 smithi133 ceph-mon[31214]: from='client.14745 v1:172.21.15.99:0/929751512' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:19:12.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:11 smithi133 ceph-mon[31214]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:12.691 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 podman[71278]: 2024-09-10 18:19:12.457228879 +0000 UTC m=+3.541027605 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:19:12.691 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 podman[71278]: 2024-09-10 18:19:12.689373165 +0000 UTC m=+3.773171891 container create 570576c412f701f4ffbb7ea57117b7ca7852756ea2050eb8d7148fb749d52622 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:12.977 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 podman[71278]: 2024-09-10 18:19:12.891886273 +0000 UTC m=+3.975684995 container init 570576c412f701f4ffbb7ea57117b7ca7852756ea2050eb8d7148fb749d52622 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:12.977 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 podman[71278]: 2024-09-10 18:19:12.895259079 +0000 UTC m=+3.979057801 container start 570576c412f701f4ffbb7ea57117b7ca7852756ea2050eb8d7148fb749d52622 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:12.977 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.899Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:19:12.977 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.899Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:19:12.977 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.900Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:19:12.977 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.900Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:19:12.977 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.900Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:19:12.977 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.900Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:19:12.977 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:19:12.977 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:19:12.978 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:19:12.978 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:19:12.978 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:19:12.978 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:19:12.978 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:19:12.978 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:19:12.978 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:19:12.979 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:19:12.979 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:19:12.979 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:19:12.979 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:19:12.979 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:19:12.979 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:19:12.979 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:19:12.979 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:19:12.979 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:19:12.979 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:19:12.979 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:19:12.979 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:19:12.980 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:19:12.980 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:19:12.980 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:19:12.980 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:19:12.980 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:19:12.980 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:19:12.980 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:19:12.980 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:19:12.980 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:19:12.980 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:19:12.980 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:19:12.980 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:19:12.980 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:19:12.981 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:19:12.981 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:19:12.981 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:19:12.981 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:19:12.981 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:19:12.981 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:19:12.981 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:19:12.981 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:19:12.981 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:19:12.981 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:19:12.981 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:19:12.981 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:19:12.981 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a[71518]: ts=2024-09-10T18:19:12.901Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:19:12.982 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 bash[71278]: 570576c412f701f4ffbb7ea57117b7ca7852756ea2050eb8d7148fb749d52622 2024-09-10T18:19:13.255 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:19:12 smithi099 systemd[1]: Started Ceph node-exporter.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:19:13.630 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:13 smithi099 ceph-mon[37557]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:13.630 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:13 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:13.630 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:13 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:13.630 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:13 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:13.630 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:13 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:13.631 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:13 smithi099 ceph-mon[27037]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:13.631 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:13 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:13.631 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:13 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:13.631 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:13 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:13.631 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:13 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:13.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:19:13 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:19:13.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:13 smithi133 ceph-mon[31214]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:13.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:13 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:13.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:13 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:13.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:13 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:13.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:13 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:14.134 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:19:14.136 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T18:19:14.241 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:13 smithi133 bash[62342]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-10T18:19:14.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:14 smithi099 ceph-mon[27037]: Deploying daemon node-exporter.b on smithi133 2024-09-10T18:19:14.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:14 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:14.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:14 smithi099 ceph-mon[37557]: Deploying daemon node-exporter.b on smithi133 2024-09-10T18:19:14.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:14 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:14.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:14 smithi133 ceph-mon[31214]: Deploying daemon node-exporter.b on smithi133 2024-09-10T18:19:14.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:14 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:14.741 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:14 smithi133 bash[62342]: Getting image source signatures 2024-09-10T18:19:14.741 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:14 smithi133 bash[62342]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T18:19:14.741 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:14 smithi133 bash[62342]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T18:19:14.741 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:14 smithi133 bash[62342]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T18:19:15.076 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-10T18:19:12.330856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":927,"num_read_kb":784,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385380,"kb_used_data":6540,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575836,"statfs":{"total":767960285184,"available":765517656064,"internally_reserved":0,"allocated":6696960,"data_stored":3631050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002241"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678517+0000","last_change":"2024-09-10T18:17:47.913353+0000","last_active":"2024-09-10T18:17:58.678517+0000","last_peered":"2024-09-10T18:17:58.678517+0000","last_clean":"2024-09-10T18:17:58.678517+0000","last_became_active":"2024-09-10T18:17:47.913079+0000","last_became_peered":"2024-09-10T18:17:47.913079+0000","last_unstale":"2024-09-10T18:17:58.678517+0000","last_undegraded":"2024-09-10T18:17:58.678517+0000","last_fullsized":"2024-09-10T18:17:58.678517+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:29.992131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647394+0000","last_change":"2024-09-10T18:17:49.922310+0000","last_active":"2024-09-10T18:17:58.647394+0000","last_peered":"2024-09-10T18:17:58.647394+0000","last_clean":"2024-09-10T18:17:58.647394+0000","last_became_active":"2024-09-10T18:17:49.922005+0000","last_became_peered":"2024-09-10T18:17:49.922005+0000","last_unstale":"2024-09-10T18:17:58.647394+0000","last_undegraded":"2024-09-10T18:17:58.647394+0000","last_fullsized":"2024-09-10T18:17:58.647394+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:39.378712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648558+0000","last_change":"2024-09-10T18:17:51.936100+0000","last_active":"2024-09-10T18:17:58.648558+0000","last_peered":"2024-09-10T18:17:58.648558+0000","last_clean":"2024-09-10T18:17:58.648558+0000","last_became_active":"2024-09-10T18:17:51.935629+0000","last_became_peered":"2024-09-10T18:17:51.935629+0000","last_unstale":"2024-09-10T18:17:58.648558+0000","last_undegraded":"2024-09-10T18:17:58.648558+0000","last_fullsized":"2024-09-10T18:17:58.648558+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:12.856098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647571+0000","last_change":"2024-09-10T18:17:53.947886+0000","last_active":"2024-09-10T18:17:58.647571+0000","last_peered":"2024-09-10T18:17:58.647571+0000","last_clean":"2024-09-10T18:17:58.647571+0000","last_became_active":"2024-09-10T18:17:53.947572+0000","last_became_peered":"2024-09-10T18:17:53.947572+0000","last_unstale":"2024-09-10T18:17:58.647571+0000","last_undegraded":"2024-09-10T18:17:58.647571+0000","last_fullsized":"2024-09-10T18:17:58.647571+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:52.841817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686377+0000","last_change":"2024-09-10T18:17:54.187299+0000","last_active":"2024-09-10T18:17:58.686377+0000","last_peered":"2024-09-10T18:17:58.686377+0000","last_clean":"2024-09-10T18:17:58.686377+0000","last_became_active":"2024-09-10T18:17:54.186708+0000","last_became_peered":"2024-09-10T18:17:54.186708+0000","last_unstale":"2024-09-10T18:17:58.686377+0000","last_undegraded":"2024-09-10T18:17:58.686377+0000","last_fullsized":"2024-09-10T18:17:58.686377+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:41.488807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686876+0000","last_change":"2024-09-10T18:17:47.912779+0000","last_active":"2024-09-10T18:17:58.686876+0000","last_peered":"2024-09-10T18:17:58.686876+0000","last_clean":"2024-09-10T18:17:58.686876+0000","last_became_active":"2024-09-10T18:17:47.912369+0000","last_became_peered":"2024-09-10T18:17:47.912369+0000","last_unstale":"2024-09-10T18:17:58.686876+0000","last_undegraded":"2024-09-10T18:17:58.686876+0000","last_fullsized":"2024-09-10T18:17:58.686876+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:19.832774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686384+0000","last_change":"2024-09-10T18:17:49.930063+0000","last_active":"2024-09-10T18:17:58.686384+0000","last_peered":"2024-09-10T18:17:58.686384+0000","last_clean":"2024-09-10T18:17:58.686384+0000","last_became_active":"2024-09-10T18:17:49.929379+0000","last_became_peered":"2024-09-10T18:17:49.929379+0000","last_unstale":"2024-09-10T18:17:58.686384+0000","last_undegraded":"2024-09-10T18:17:58.686384+0000","last_fullsized":"2024-09-10T18:17:58.686384+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:24.111351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647374+0000","last_change":"2024-09-10T18:17:51.946448+0000","last_active":"2024-09-10T18:17:58.647374+0000","last_peered":"2024-09-10T18:17:58.647374+0000","last_clean":"2024-09-10T18:17:58.647374+0000","last_became_active":"2024-09-10T18:17:51.946170+0000","last_became_peered":"2024-09-10T18:17:51.946170+0000","last_unstale":"2024-09-10T18:17:58.647374+0000","last_undegraded":"2024-09-10T18:17:58.647374+0000","last_fullsized":"2024-09-10T18:17:58.647374+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:04.898014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961659+0000","last_change":"2024-09-10T18:17:47.914996+0000","last_active":"2024-09-10T18:17:58.961659+0000","last_peered":"2024-09-10T18:17:58.961659+0000","last_clean":"2024-09-10T18:17:58.961659+0000","last_became_active":"2024-09-10T18:17:47.914722+0000","last_became_peered":"2024-09-10T18:17:47.914722+0000","last_unstale":"2024-09-10T18:17:58.961659+0000","last_undegraded":"2024-09-10T18:17:58.961659+0000","last_fullsized":"2024-09-10T18:17:58.961659+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:24.340748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647745+0000","last_change":"2024-09-10T18:17:49.926098+0000","last_active":"2024-09-10T18:17:58.647745+0000","last_peered":"2024-09-10T18:17:58.647745+0000","last_clean":"2024-09-10T18:17:58.647745+0000","last_became_active":"2024-09-10T18:17:49.925857+0000","last_became_peered":"2024-09-10T18:17:49.925857+0000","last_unstale":"2024-09-10T18:17:58.647745+0000","last_undegraded":"2024-09-10T18:17:58.647745+0000","last_fullsized":"2024-09-10T18:17:58.647745+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:25.710523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961398+0000","last_change":"2024-09-10T18:17:51.941125+0000","last_active":"2024-09-10T18:17:58.961398+0000","last_peered":"2024-09-10T18:17:58.961398+0000","last_clean":"2024-09-10T18:17:58.961398+0000","last_became_active":"2024-09-10T18:17:51.940227+0000","last_became_peered":"2024-09-10T18:17:51.940227+0000","last_unstale":"2024-09-10T18:17:58.961398+0000","last_undegraded":"2024-09-10T18:17:58.961398+0000","last_fullsized":"2024-09-10T18:17:58.961398+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:25.218335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.677945+0000","last_change":"2024-09-10T18:17:53.955325+0000","last_active":"2024-09-10T18:17:58.677945+0000","last_peered":"2024-09-10T18:17:58.677945+0000","last_clean":"2024-09-10T18:17:58.677945+0000","last_became_active":"2024-09-10T18:17:53.955034+0000","last_became_peered":"2024-09-10T18:17:53.955034+0000","last_unstale":"2024-09-10T18:17:58.677945+0000","last_undegraded":"2024-09-10T18:17:58.677945+0000","last_fullsized":"2024-09-10T18:17:58.677945+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:49.821426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961764+0000","last_change":"2024-09-10T18:17:47.920214+0000","last_active":"2024-09-10T18:17:58.961764+0000","last_peered":"2024-09-10T18:17:58.961764+0000","last_clean":"2024-09-10T18:17:58.961764+0000","last_became_active":"2024-09-10T18:17:47.919890+0000","last_became_peered":"2024-09-10T18:17:47.919890+0000","last_unstale":"2024-09-10T18:17:58.961764+0000","last_undegraded":"2024-09-10T18:17:58.961764+0000","last_fullsized":"2024-09-10T18:17:58.961764+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:13.428652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647760+0000","last_change":"2024-09-10T18:17:49.927753+0000","last_active":"2024-09-10T18:17:58.647760+0000","last_peered":"2024-09-10T18:17:58.647760+0000","last_clean":"2024-09-10T18:17:58.647760+0000","last_became_active":"2024-09-10T18:17:49.927476+0000","last_became_peered":"2024-09-10T18:17:49.927476+0000","last_unstale":"2024-09-10T18:17:58.647760+0000","last_undegraded":"2024-09-10T18:17:58.647760+0000","last_fullsized":"2024-09-10T18:17:58.647760+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:24.357068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961488+0000","last_change":"2024-09-10T18:17:51.940403+0000","last_active":"2024-09-10T18:17:58.961488+0000","last_peered":"2024-09-10T18:17:58.961488+0000","last_clean":"2024-09-10T18:17:58.961488+0000","last_became_active":"2024-09-10T18:17:51.939298+0000","last_became_peered":"2024-09-10T18:17:51.939298+0000","last_unstale":"2024-09-10T18:17:58.961488+0000","last_undegraded":"2024-09-10T18:17:58.961488+0000","last_fullsized":"2024-09-10T18:17:58.961488+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:58.780971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961190+0000","last_change":"2024-09-10T18:17:53.949036+0000","last_active":"2024-09-10T18:17:58.961190+0000","last_peered":"2024-09-10T18:17:58.961190+0000","last_clean":"2024-09-10T18:17:58.961190+0000","last_became_active":"2024-09-10T18:17:53.948754+0000","last_became_peered":"2024-09-10T18:17:53.948754+0000","last_unstale":"2024-09-10T18:17:58.961190+0000","last_undegraded":"2024-09-10T18:17:58.961190+0000","last_fullsized":"2024-09-10T18:17:58.961190+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:41.493158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648190+0000","last_change":"2024-09-10T18:17:54.187718+0000","last_active":"2024-09-10T18:17:58.648190+0000","last_peered":"2024-09-10T18:17:58.648190+0000","last_clean":"2024-09-10T18:17:58.648190+0000","last_became_active":"2024-09-10T18:17:54.187436+0000","last_became_peered":"2024-09-10T18:17:54.187436+0000","last_unstale":"2024-09-10T18:17:58.648190+0000","last_undegraded":"2024-09-10T18:17:58.648190+0000","last_fullsized":"2024-09-10T18:17:58.648190+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:05.281186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"52'1","reported_seq":27,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678613+0000","last_change":"2024-09-10T18:17:47.912942+0000","last_active":"2024-09-10T18:17:58.678613+0000","last_peered":"2024-09-10T18:17:58.678613+0000","last_clean":"2024-09-10T18:17:58.678613+0000","last_became_active":"2024-09-10T18:17:47.912638+0000","last_became_peered":"2024-09-10T18:17:47.912638+0000","last_unstale":"2024-09-10T18:17:58.678613+0000","last_undegraded":"2024-09-10T18:17:58.678613+0000","last_fullsized":"2024-09-10T18:17:58.678613+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:12.335422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.649040+0000","last_change":"2024-09-10T18:17:49.929932+0000","last_active":"2024-09-10T18:17:58.649040+0000","last_peered":"2024-09-10T18:17:58.649040+0000","last_clean":"2024-09-10T18:17:58.649040+0000","last_became_active":"2024-09-10T18:17:49.929643+0000","last_became_peered":"2024-09-10T18:17:49.929643+0000","last_unstale":"2024-09-10T18:17:58.649040+0000","last_undegraded":"2024-09-10T18:17:58.649040+0000","last_fullsized":"2024-09-10T18:17:58.649040+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:56.577829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648075+0000","last_change":"2024-09-10T18:17:51.940485+0000","last_active":"2024-09-10T18:17:58.648075+0000","last_peered":"2024-09-10T18:17:58.648075+0000","last_clean":"2024-09-10T18:17:58.648075+0000","last_became_active":"2024-09-10T18:17:51.940168+0000","last_became_peered":"2024-09-10T18:17:51.940168+0000","last_unstale":"2024-09-10T18:17:58.648075+0000","last_undegraded":"2024-09-10T18:17:58.648075+0000","last_fullsized":"2024-09-10T18:17:58.648075+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:48.045412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686873+0000","last_change":"2024-09-10T18:17:54.186437+0000","last_active":"2024-09-10T18:17:58.686873+0000","last_peered":"2024-09-10T18:17:58.686873+0000","last_clean":"2024-09-10T18:17:58.686873+0000","last_became_active":"2024-09-10T18:17:54.186098+0000","last_became_peered":"2024-09-10T18:17:54.186098+0000","last_unstale":"2024-09-10T18:17:58.686873+0000","last_undegraded":"2024-09-10T18:17:58.686873+0000","last_fullsized":"2024-09-10T18:17:58.686873+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:48:09.715096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647812+0000","last_change":"2024-09-10T18:17:47.917385+0000","last_active":"2024-09-10T18:17:58.647812+0000","last_peered":"2024-09-10T18:17:58.647812+0000","last_clean":"2024-09-10T18:17:58.647812+0000","last_became_active":"2024-09-10T18:17:47.917131+0000","last_became_peered":"2024-09-10T18:17:47.917131+0000","last_unstale":"2024-09-10T18:17:58.647812+0000","last_undegraded":"2024-09-10T18:17:58.647812+0000","last_fullsized":"2024-09-10T18:17:58.647812+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:19.619168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686659+0000","last_change":"2024-09-10T18:17:49.928588+0000","last_active":"2024-09-10T18:17:58.686659+0000","last_peered":"2024-09-10T18:17:58.686659+0000","last_clean":"2024-09-10T18:17:58.686659+0000","last_became_active":"2024-09-10T18:17:49.928310+0000","last_became_peered":"2024-09-10T18:17:49.928310+0000","last_unstale":"2024-09-10T18:17:58.686659+0000","last_undegraded":"2024-09-10T18:17:58.686659+0000","last_fullsized":"2024-09-10T18:17:58.686659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:12.617190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647893+0000","last_change":"2024-09-10T18:17:51.947269+0000","last_active":"2024-09-10T18:17:58.647893+0000","last_peered":"2024-09-10T18:17:58.647893+0000","last_clean":"2024-09-10T18:17:58.647893+0000","last_became_active":"2024-09-10T18:17:51.946866+0000","last_became_peered":"2024-09-10T18:17:51.946866+0000","last_unstale":"2024-09-10T18:17:58.647893+0000","last_undegraded":"2024-09-10T18:17:58.647893+0000","last_fullsized":"2024-09-10T18:17:58.647893+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:49.464097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962136+0000","last_change":"2024-09-10T18:17:53.948845+0000","last_active":"2024-09-10T18:17:58.962136+0000","last_peered":"2024-09-10T18:17:58.962136+0000","last_clean":"2024-09-10T18:17:58.962136+0000","last_became_active":"2024-09-10T18:17:53.948570+0000","last_became_peered":"2024-09-10T18:17:53.948570+0000","last_unstale":"2024-09-10T18:17:58.962136+0000","last_undegraded":"2024-09-10T18:17:58.962136+0000","last_fullsized":"2024-09-10T18:17:58.962136+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:51.344212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648065+0000","last_change":"2024-09-10T18:17:47.919574+0000","last_active":"2024-09-10T18:17:58.648065+0000","last_peered":"2024-09-10T18:17:58.648065+0000","last_clean":"2024-09-10T18:17:58.648065+0000","last_became_active":"2024-09-10T18:17:47.919207+0000","last_became_peered":"2024-09-10T18:17:47.919207+0000","last_unstale":"2024-09-10T18:17:58.648065+0000","last_undegraded":"2024-09-10T18:17:58.648065+0000","last_fullsized":"2024-09-10T18:17:58.648065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:52.343060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678478+0000","last_change":"2024-09-10T18:17:50.099083+0000","last_active":"2024-09-10T18:17:58.678478+0000","last_peered":"2024-09-10T18:17:58.678478+0000","last_clean":"2024-09-10T18:17:58.678478+0000","last_became_active":"2024-09-10T18:17:50.098823+0000","last_became_peered":"2024-09-10T18:17:50.098823+0000","last_unstale":"2024-09-10T18:17:58.678478+0000","last_undegraded":"2024-09-10T18:17:58.678478+0000","last_fullsized":"2024-09-10T18:17:58.678478+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:16.154758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107613+0000","last_change":"2024-09-10T18:17:51.945246+0000","last_active":"2024-09-10T18:19:01.107613+0000","last_peered":"2024-09-10T18:19:01.107613+0000","last_clean":"2024-09-10T18:19:01.107613+0000","last_became_active":"2024-09-10T18:17:51.943976+0000","last_became_peered":"2024-09-10T18:17:51.943976+0000","last_unstale":"2024-09-10T18:19:01.107613+0000","last_undegraded":"2024-09-10T18:19:01.107613+0000","last_fullsized":"2024-09-10T18:19:01.107613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:51.322968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648790+0000","last_change":"2024-09-10T18:17:49.929859+0000","last_active":"2024-09-10T18:17:58.648790+0000","last_peered":"2024-09-10T18:17:58.648790+0000","last_clean":"2024-09-10T18:17:58.648790+0000","last_became_active":"2024-09-10T18:17:49.929299+0000","last_became_peered":"2024-09-10T18:17:49.929299+0000","last_unstale":"2024-09-10T18:17:58.648790+0000","last_undegraded":"2024-09-10T18:17:58.648790+0000","last_fullsized":"2024-09-10T18:17:58.648790+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:54:02.276340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686278+0000","last_change":"2024-09-10T18:17:47.911501+0000","last_active":"2024-09-10T18:17:58.686278+0000","last_peered":"2024-09-10T18:17:58.686278+0000","last_clean":"2024-09-10T18:17:58.686278+0000","last_became_active":"2024-09-10T18:17:47.911224+0000","last_became_peered":"2024-09-10T18:17:47.911224+0000","last_unstale":"2024-09-10T18:17:58.686278+0000","last_undegraded":"2024-09-10T18:17:58.686278+0000","last_fullsized":"2024-09-10T18:17:58.686278+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:17.441640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107613+0000","last_change":"2024-09-10T18:17:51.943508+0000","last_active":"2024-09-10T18:19:01.107613+0000","last_peered":"2024-09-10T18:19:01.107613+0000","last_clean":"2024-09-10T18:19:01.107613+0000","last_became_active":"2024-09-10T18:17:51.943208+0000","last_became_peered":"2024-09-10T18:17:51.943208+0000","last_unstale":"2024-09-10T18:19:01.107613+0000","last_undegraded":"2024-09-10T18:19:01.107613+0000","last_fullsized":"2024-09-10T18:19:01.107613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:28.575117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961278+0000","last_change":"2024-09-10T18:17:53.955381+0000","last_active":"2024-09-10T18:17:58.961278+0000","last_peered":"2024-09-10T18:17:58.961278+0000","last_clean":"2024-09-10T18:17:58.961278+0000","last_became_active":"2024-09-10T18:17:53.955112+0000","last_became_peered":"2024-09-10T18:17:53.955112+0000","last_unstale":"2024-09-10T18:17:58.961278+0000","last_undegraded":"2024-09-10T18:17:58.961278+0000","last_fullsized":"2024-09-10T18:17:58.961278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:56.887383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686091+0000","last_change":"2024-09-10T18:17:50.098970+0000","last_active":"2024-09-10T18:17:58.686091+0000","last_peered":"2024-09-10T18:17:58.686091+0000","last_clean":"2024-09-10T18:17:58.686091+0000","last_became_active":"2024-09-10T18:17:50.098718+0000","last_became_peered":"2024-09-10T18:17:50.098718+0000","last_unstale":"2024-09-10T18:17:58.686091+0000","last_undegraded":"2024-09-10T18:17:58.686091+0000","last_fullsized":"2024-09-10T18:17:58.686091+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:17.432478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.687005+0000","last_change":"2024-09-10T18:17:47.912791+0000","last_active":"2024-09-10T18:17:58.687005+0000","last_peered":"2024-09-10T18:17:58.687005+0000","last_clean":"2024-09-10T18:17:58.687005+0000","last_became_active":"2024-09-10T18:17:47.912537+0000","last_became_peered":"2024-09-10T18:17:47.912537+0000","last_unstale":"2024-09-10T18:17:58.687005+0000","last_undegraded":"2024-09-10T18:17:58.687005+0000","last_fullsized":"2024-09-10T18:17:58.687005+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:28.199019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647423+0000","last_change":"2024-09-10T18:17:51.948141+0000","last_active":"2024-09-10T18:17:58.647423+0000","last_peered":"2024-09-10T18:17:58.647423+0000","last_clean":"2024-09-10T18:17:58.647423+0000","last_became_active":"2024-09-10T18:17:51.947662+0000","last_became_peered":"2024-09-10T18:17:51.947662+0000","last_unstale":"2024-09-10T18:17:58.647423+0000","last_undegraded":"2024-09-10T18:17:58.647423+0000","last_fullsized":"2024-09-10T18:17:58.647423+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:54.239287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647721+0000","last_change":"2024-09-10T18:17:53.947259+0000","last_active":"2024-09-10T18:17:58.647721+0000","last_peered":"2024-09-10T18:17:58.647721+0000","last_clean":"2024-09-10T18:17:58.647721+0000","last_became_active":"2024-09-10T18:17:53.946967+0000","last_became_peered":"2024-09-10T18:17:53.946967+0000","last_unstale":"2024-09-10T18:17:58.647721+0000","last_undegraded":"2024-09-10T18:17:58.647721+0000","last_fullsized":"2024-09-10T18:17:58.647721+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:00.131810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647497+0000","last_change":"2024-09-10T18:17:49.920000+0000","last_active":"2024-09-10T18:17:58.647497+0000","last_peered":"2024-09-10T18:17:58.647497+0000","last_clean":"2024-09-10T18:17:58.647497+0000","last_became_active":"2024-09-10T18:17:49.919667+0000","last_became_peered":"2024-09-10T18:17:49.919667+0000","last_unstale":"2024-09-10T18:17:58.647497+0000","last_undegraded":"2024-09-10T18:17:58.647497+0000","last_fullsized":"2024-09-10T18:17:58.647497+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:09.676833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678607+0000","last_change":"2024-09-10T18:17:47.907334+0000","last_active":"2024-09-10T18:17:58.678607+0000","last_peered":"2024-09-10T18:17:58.678607+0000","last_clean":"2024-09-10T18:17:58.678607+0000","last_became_active":"2024-09-10T18:17:47.907070+0000","last_became_peered":"2024-09-10T18:17:47.907070+0000","last_unstale":"2024-09-10T18:17:58.678607+0000","last_undegraded":"2024-09-10T18:17:58.678607+0000","last_fullsized":"2024-09-10T18:17:58.678607+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:14.976361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107815+0000","last_change":"2024-09-10T18:17:51.945298+0000","last_active":"2024-09-10T18:19:01.107815+0000","last_peered":"2024-09-10T18:19:01.107815+0000","last_clean":"2024-09-10T18:19:01.107815+0000","last_became_active":"2024-09-10T18:17:51.944841+0000","last_became_peered":"2024-09-10T18:17:51.944841+0000","last_unstale":"2024-09-10T18:19:01.107815+0000","last_undegraded":"2024-09-10T18:19:01.107815+0000","last_fullsized":"2024-09-10T18:19:01.107815+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:41.514771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961715+0000","last_change":"2024-09-10T18:17:54.190656+0000","last_active":"2024-09-10T18:17:58.961715+0000","last_peered":"2024-09-10T18:17:58.961715+0000","last_clean":"2024-09-10T18:17:58.961715+0000","last_became_active":"2024-09-10T18:17:54.189698+0000","last_became_peered":"2024-09-10T18:17:54.189698+0000","last_unstale":"2024-09-10T18:17:58.961715+0000","last_undegraded":"2024-09-10T18:17:58.961715+0000","last_fullsized":"2024-09-10T18:17:58.961715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:40.652122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648965+0000","last_change":"2024-09-10T18:17:49.927162+0000","last_active":"2024-09-10T18:17:58.648965+0000","last_peered":"2024-09-10T18:17:58.648965+0000","last_clean":"2024-09-10T18:17:58.648965+0000","last_became_active":"2024-09-10T18:17:49.926851+0000","last_became_peered":"2024-09-10T18:17:49.926851+0000","last_unstale":"2024-09-10T18:17:58.648965+0000","last_undegraded":"2024-09-10T18:17:58.648965+0000","last_fullsized":"2024-09-10T18:17:58.648965+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:48.113651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961727+0000","last_change":"2024-09-10T18:17:47.913460+0000","last_active":"2024-09-10T18:17:58.961727+0000","last_peered":"2024-09-10T18:17:58.961727+0000","last_clean":"2024-09-10T18:17:58.961727+0000","last_became_active":"2024-09-10T18:17:47.913189+0000","last_became_peered":"2024-09-10T18:17:47.913189+0000","last_unstale":"2024-09-10T18:17:58.961727+0000","last_undegraded":"2024-09-10T18:17:58.961727+0000","last_fullsized":"2024-09-10T18:17:58.961727+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:22.970274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107719+0000","last_change":"2024-09-10T18:17:51.939209+0000","last_active":"2024-09-10T18:19:01.107719+0000","last_peered":"2024-09-10T18:19:01.107719+0000","last_clean":"2024-09-10T18:19:01.107719+0000","last_became_active":"2024-09-10T18:17:51.938931+0000","last_became_peered":"2024-09-10T18:17:51.938931+0000","last_unstale":"2024-09-10T18:19:01.107719+0000","last_undegraded":"2024-09-10T18:19:01.107719+0000","last_fullsized":"2024-09-10T18:19:01.107719+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:44.974616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678120+0000","last_change":"2024-09-10T18:17:53.953867+0000","last_active":"2024-09-10T18:17:58.678120+0000","last_peered":"2024-09-10T18:17:58.678120+0000","last_clean":"2024-09-10T18:17:58.678120+0000","last_became_active":"2024-09-10T18:17:53.953069+0000","last_became_peered":"2024-09-10T18:17:53.953069+0000","last_unstale":"2024-09-10T18:17:58.678120+0000","last_undegraded":"2024-09-10T18:17:58.678120+0000","last_fullsized":"2024-09-10T18:17:58.678120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:31.926307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678431+0000","last_change":"2024-09-10T18:17:50.099431+0000","last_active":"2024-09-10T18:17:58.678431+0000","last_peered":"2024-09-10T18:17:58.678431+0000","last_clean":"2024-09-10T18:17:58.678431+0000","last_became_active":"2024-09-10T18:17:50.099142+0000","last_became_peered":"2024-09-10T18:17:50.099142+0000","last_unstale":"2024-09-10T18:17:58.678431+0000","last_undegraded":"2024-09-10T18:17:58.678431+0000","last_fullsized":"2024-09-10T18:17:58.678431+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:58.443405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648761+0000","last_change":"2024-09-10T18:17:47.915490+0000","last_active":"2024-09-10T18:17:58.648761+0000","last_peered":"2024-09-10T18:17:58.648761+0000","last_clean":"2024-09-10T18:17:58.648761+0000","last_became_active":"2024-09-10T18:17:47.915011+0000","last_became_peered":"2024-09-10T18:17:47.915011+0000","last_unstale":"2024-09-10T18:17:58.648761+0000","last_undegraded":"2024-09-10T18:17:58.648761+0000","last_fullsized":"2024-09-10T18:17:58.648761+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:41.633692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962131+0000","last_change":"2024-09-10T18:17:51.944137+0000","last_active":"2024-09-10T18:17:58.962131+0000","last_peered":"2024-09-10T18:17:58.962131+0000","last_clean":"2024-09-10T18:17:58.962131+0000","last_became_active":"2024-09-10T18:17:51.943793+0000","last_became_peered":"2024-09-10T18:17:51.943793+0000","last_unstale":"2024-09-10T18:17:58.962131+0000","last_undegraded":"2024-09-10T18:17:58.962131+0000","last_fullsized":"2024-09-10T18:17:58.962131+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:30.728743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647456+0000","last_change":"2024-09-10T18:17:54.186331+0000","last_active":"2024-09-10T18:17:58.647456+0000","last_peered":"2024-09-10T18:17:58.647456+0000","last_clean":"2024-09-10T18:17:58.647456+0000","last_became_active":"2024-09-10T18:17:54.185947+0000","last_became_peered":"2024-09-10T18:17:54.185947+0000","last_unstale":"2024-09-10T18:17:58.647456+0000","last_undegraded":"2024-09-10T18:17:58.647456+0000","last_fullsized":"2024-09-10T18:17:58.647456+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:18.108739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107719+0000","last_change":"2024-09-10T18:17:49.931497+0000","last_active":"2024-09-10T18:19:01.107719+0000","last_peered":"2024-09-10T18:19:01.107719+0000","last_clean":"2024-09-10T18:19:01.107719+0000","last_became_active":"2024-09-10T18:17:49.931193+0000","last_became_peered":"2024-09-10T18:17:49.931193+0000","last_unstale":"2024-09-10T18:19:01.107719+0000","last_undegraded":"2024-09-10T18:19:01.107719+0000","last_fullsized":"2024-09-10T18:19:01.107719+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:18.393114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647977+0000","last_change":"2024-09-10T18:17:47.914843+0000","last_active":"2024-09-10T18:17:58.647977+0000","last_peered":"2024-09-10T18:17:58.647977+0000","last_clean":"2024-09-10T18:17:58.647977+0000","last_became_active":"2024-09-10T18:17:47.914569+0000","last_became_peered":"2024-09-10T18:17:47.914569+0000","last_unstale":"2024-09-10T18:17:58.647977+0000","last_undegraded":"2024-09-10T18:17:58.647977+0000","last_fullsized":"2024-09-10T18:17:58.647977+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:19.792761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961840+0000","last_change":"2024-09-10T18:17:51.933437+0000","last_active":"2024-09-10T18:17:58.961840+0000","last_peered":"2024-09-10T18:17:58.961840+0000","last_clean":"2024-09-10T18:17:58.961840+0000","last_became_active":"2024-09-10T18:17:47.915524+0000","last_became_peered":"2024-09-10T18:17:47.915524+0000","last_unstale":"2024-09-10T18:17:58.961840+0000","last_undegraded":"2024-09-10T18:17:58.961840+0000","last_fullsized":"2024-09-10T18:17:58.961840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:40:32.994132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043679999999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678401+0000","last_change":"2024-09-10T18:17:51.943377+0000","last_active":"2024-09-10T18:17:58.678401+0000","last_peered":"2024-09-10T18:17:58.678401+0000","last_clean":"2024-09-10T18:17:58.678401+0000","last_became_active":"2024-09-10T18:17:51.942963+0000","last_became_peered":"2024-09-10T18:17:51.942963+0000","last_unstale":"2024-09-10T18:17:58.678401+0000","last_undegraded":"2024-09-10T18:17:58.678401+0000","last_fullsized":"2024-09-10T18:17:58.678401+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:02.069882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686700+0000","last_change":"2024-09-10T18:17:53.945868+0000","last_active":"2024-09-10T18:17:58.686700+0000","last_peered":"2024-09-10T18:17:58.686700+0000","last_clean":"2024-09-10T18:17:58.686700+0000","last_became_active":"2024-09-10T18:17:53.945559+0000","last_became_peered":"2024-09-10T18:17:53.945559+0000","last_unstale":"2024-09-10T18:17:58.686700+0000","last_undegraded":"2024-09-10T18:17:58.686700+0000","last_fullsized":"2024-09-10T18:17:58.686700+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:14.679190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648684+0000","last_change":"2024-09-10T18:17:49.923094+0000","last_active":"2024-09-10T18:17:58.648684+0000","last_peered":"2024-09-10T18:17:58.648684+0000","last_clean":"2024-09-10T18:17:58.648684+0000","last_became_active":"2024-09-10T18:17:49.922256+0000","last_became_peered":"2024-09-10T18:17:49.922256+0000","last_unstale":"2024-09-10T18:17:58.648684+0000","last_undegraded":"2024-09-10T18:17:58.648684+0000","last_fullsized":"2024-09-10T18:17:58.648684+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:53.639177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647663+0000","last_change":"2024-09-10T18:17:47.915334+0000","last_active":"2024-09-10T18:17:58.647663+0000","last_peered":"2024-09-10T18:17:58.647663+0000","last_clean":"2024-09-10T18:17:58.647663+0000","last_became_active":"2024-09-10T18:17:47.915087+0000","last_became_peered":"2024-09-10T18:17:47.915087+0000","last_unstale":"2024-09-10T18:17:58.647663+0000","last_undegraded":"2024-09-10T18:17:58.647663+0000","last_fullsized":"2024-09-10T18:17:58.647663+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:09.868408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648126+0000","last_change":"2024-09-10T18:17:51.930420+0000","last_active":"2024-09-10T18:17:58.648126+0000","last_peered":"2024-09-10T18:17:58.648126+0000","last_clean":"2024-09-10T18:17:58.648126+0000","last_became_active":"2024-09-10T18:17:47.908038+0000","last_became_peered":"2024-09-10T18:17:47.908038+0000","last_unstale":"2024-09-10T18:17:58.648126+0000","last_undegraded":"2024-09-10T18:17:58.648126+0000","last_fullsized":"2024-09-10T18:17:58.648126+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:43.957190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011789140000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648672+0000","last_change":"2024-09-10T18:17:51.939553+0000","last_active":"2024-09-10T18:17:58.648672+0000","last_peered":"2024-09-10T18:17:58.648672+0000","last_clean":"2024-09-10T18:17:58.648672+0000","last_became_active":"2024-09-10T18:17:51.939291+0000","last_became_peered":"2024-09-10T18:17:51.939291+0000","last_unstale":"2024-09-10T18:17:58.648672+0000","last_undegraded":"2024-09-10T18:17:58.648672+0000","last_fullsized":"2024-09-10T18:17:58.648672+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:54.887490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962883+0000","last_change":"2024-09-10T18:17:54.189214+0000","last_active":"2024-09-10T18:17:58.962883+0000","last_peered":"2024-09-10T18:17:58.962883+0000","last_clean":"2024-09-10T18:17:58.962883+0000","last_became_active":"2024-09-10T18:17:54.188939+0000","last_became_peered":"2024-09-10T18:17:54.188939+0000","last_unstale":"2024-09-10T18:17:58.962883+0000","last_undegraded":"2024-09-10T18:17:58.962883+0000","last_fullsized":"2024-09-10T18:17:58.962883+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:44.556501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678534+0000","last_change":"2024-09-10T18:17:49.923899+0000","last_active":"2024-09-10T18:17:58.678534+0000","last_peered":"2024-09-10T18:17:58.678534+0000","last_clean":"2024-09-10T18:17:58.678534+0000","last_became_active":"2024-09-10T18:17:49.923569+0000","last_became_peered":"2024-09-10T18:17:49.923569+0000","last_unstale":"2024-09-10T18:17:58.678534+0000","last_undegraded":"2024-09-10T18:17:58.678534+0000","last_fullsized":"2024-09-10T18:17:58.678534+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:53.078822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678003+0000","last_change":"2024-09-10T18:17:47.909157+0000","last_active":"2024-09-10T18:17:58.678003+0000","last_peered":"2024-09-10T18:17:58.678003+0000","last_clean":"2024-09-10T18:17:58.678003+0000","last_became_active":"2024-09-10T18:17:47.908868+0000","last_became_peered":"2024-09-10T18:17:47.908868+0000","last_unstale":"2024-09-10T18:17:58.678003+0000","last_undegraded":"2024-09-10T18:17:58.678003+0000","last_fullsized":"2024-09-10T18:17:58.678003+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:10.715648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"60'5","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:07.002311+0000","last_change":"2024-09-10T18:17:51.929853+0000","last_active":"2024-09-10T18:19:07.002311+0000","last_peered":"2024-09-10T18:19:07.002311+0000","last_clean":"2024-09-10T18:19:07.002311+0000","last_became_active":"2024-09-10T18:17:47.918342+0000","last_became_peered":"2024-09-10T18:17:47.918342+0000","last_unstale":"2024-09-10T18:19:07.002311+0000","last_undegraded":"2024-09-10T18:19:07.002311+0000","last_fullsized":"2024-09-10T18:19:07.002311+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:34.520922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035361600000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962287+0000","last_change":"2024-09-10T18:17:51.943234+0000","last_active":"2024-09-10T18:17:58.962287+0000","last_peered":"2024-09-10T18:17:58.962287+0000","last_clean":"2024-09-10T18:17:58.962287+0000","last_became_active":"2024-09-10T18:17:51.941422+0000","last_became_peered":"2024-09-10T18:17:51.941422+0000","last_unstale":"2024-09-10T18:17:58.962287+0000","last_undegraded":"2024-09-10T18:17:58.962287+0000","last_fullsized":"2024-09-10T18:17:58.962287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:40.453088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648326+0000","last_change":"2024-09-10T18:17:53.953464+0000","last_active":"2024-09-10T18:17:58.648326+0000","last_peered":"2024-09-10T18:17:58.648326+0000","last_clean":"2024-09-10T18:17:58.648326+0000","last_became_active":"2024-09-10T18:17:53.953064+0000","last_became_peered":"2024-09-10T18:17:53.953064+0000","last_unstale":"2024-09-10T18:17:58.648326+0000","last_undegraded":"2024-09-10T18:17:58.648326+0000","last_fullsized":"2024-09-10T18:17:58.648326+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:59.398334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961861+0000","last_change":"2024-09-10T18:17:50.099216+0000","last_active":"2024-09-10T18:17:58.961861+0000","last_peered":"2024-09-10T18:17:58.961861+0000","last_clean":"2024-09-10T18:17:58.961861+0000","last_became_active":"2024-09-10T18:17:50.098878+0000","last_became_peered":"2024-09-10T18:17:50.098878+0000","last_unstale":"2024-09-10T18:17:58.961861+0000","last_undegraded":"2024-09-10T18:17:58.961861+0000","last_fullsized":"2024-09-10T18:17:58.961861+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:11.706968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686495+0000","last_change":"2024-09-10T18:17:47.912779+0000","last_active":"2024-09-10T18:17:58.686495+0000","last_peered":"2024-09-10T18:17:58.686495+0000","last_clean":"2024-09-10T18:17:58.686495+0000","last_became_active":"2024-09-10T18:17:47.912271+0000","last_became_peered":"2024-09-10T18:17:47.912271+0000","last_unstale":"2024-09-10T18:17:58.686495+0000","last_undegraded":"2024-09-10T18:17:58.686495+0000","last_fullsized":"2024-09-10T18:17:58.686495+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:56.789693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:18:24.365476+0000","last_change":"2024-09-10T18:17:33.855354+0000","last_active":"2024-09-10T18:18:24.365476+0000","last_peered":"2024-09-10T18:18:24.365476+0000","last_clean":"2024-09-10T18:18:24.365476+0000","last_became_active":"2024-09-10T18:17:33.849783+0000","last_became_peered":"2024-09-10T18:17:33.849783+0000","last_unstale":"2024-09-10T18:18:24.365476+0000","last_undegraded":"2024-09-10T18:18:24.365476+0000","last_fullsized":"2024-09-10T18:18:24.365476+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:15:42.260515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:15:42.260515+0000","last_clean_scrub_stamp":"2024-09-10T18:15:42.260515+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:57.467633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962261+0000","last_change":"2024-09-10T18:17:51.939177+0000","last_active":"2024-09-10T18:17:58.962261+0000","last_peered":"2024-09-10T18:17:58.962261+0000","last_clean":"2024-09-10T18:17:58.962261+0000","last_became_active":"2024-09-10T18:17:51.938805+0000","last_became_peered":"2024-09-10T18:17:51.938805+0000","last_unstale":"2024-09-10T18:17:58.962261+0000","last_undegraded":"2024-09-10T18:17:58.962261+0000","last_fullsized":"2024-09-10T18:17:58.962261+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:41.738130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962102+0000","last_change":"2024-09-10T18:17:53.948344+0000","last_active":"2024-09-10T18:17:58.962102+0000","last_peered":"2024-09-10T18:17:58.962102+0000","last_clean":"2024-09-10T18:17:58.962102+0000","last_became_active":"2024-09-10T18:17:53.947663+0000","last_became_peered":"2024-09-10T18:17:53.947663+0000","last_unstale":"2024-09-10T18:17:58.962102+0000","last_undegraded":"2024-09-10T18:17:58.962102+0000","last_fullsized":"2024-09-10T18:17:58.962102+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:08.629389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648272+0000","last_change":"2024-09-10T18:17:49.926517+0000","last_active":"2024-09-10T18:17:58.648272+0000","last_peered":"2024-09-10T18:17:58.648272+0000","last_clean":"2024-09-10T18:17:58.648272+0000","last_became_active":"2024-09-10T18:17:49.926265+0000","last_became_peered":"2024-09-10T18:17:49.926265+0000","last_unstale":"2024-09-10T18:17:58.648272+0000","last_undegraded":"2024-09-10T18:17:58.648272+0000","last_fullsized":"2024-09-10T18:17:58.648272+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:08.568044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647587+0000","last_change":"2024-09-10T18:17:47.912442+0000","last_active":"2024-09-10T18:17:58.647587+0000","last_peered":"2024-09-10T18:17:58.647587+0000","last_clean":"2024-09-10T18:17:58.647587+0000","last_became_active":"2024-09-10T18:17:47.912125+0000","last_became_peered":"2024-09-10T18:17:47.912125+0000","last_unstale":"2024-09-10T18:17:58.647587+0000","last_undegraded":"2024-09-10T18:17:58.647587+0000","last_fullsized":"2024-09-10T18:17:58.647587+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:13:44.304839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962094+0000","last_change":"2024-09-10T18:17:51.941024+0000","last_active":"2024-09-10T18:17:58.962094+0000","last_peered":"2024-09-10T18:17:58.962094+0000","last_clean":"2024-09-10T18:17:58.962094+0000","last_became_active":"2024-09-10T18:17:51.940713+0000","last_became_peered":"2024-09-10T18:17:51.940713+0000","last_unstale":"2024-09-10T18:17:58.962094+0000","last_undegraded":"2024-09-10T18:17:58.962094+0000","last_fullsized":"2024-09-10T18:17:58.962094+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:20.919047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686451+0000","last_change":"2024-09-10T18:17:54.186836+0000","last_active":"2024-09-10T18:17:58.686451+0000","last_peered":"2024-09-10T18:17:58.686451+0000","last_clean":"2024-09-10T18:17:58.686451+0000","last_became_active":"2024-09-10T18:17:54.186071+0000","last_became_peered":"2024-09-10T18:17:54.186071+0000","last_unstale":"2024-09-10T18:17:58.686451+0000","last_undegraded":"2024-09-10T18:17:58.686451+0000","last_fullsized":"2024-09-10T18:17:58.686451+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:50.730457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648161+0000","last_change":"2024-09-10T18:17:49.922410+0000","last_active":"2024-09-10T18:17:58.648161+0000","last_peered":"2024-09-10T18:17:58.648161+0000","last_clean":"2024-09-10T18:17:58.648161+0000","last_became_active":"2024-09-10T18:17:49.921884+0000","last_became_peered":"2024-09-10T18:17:49.921884+0000","last_unstale":"2024-09-10T18:17:58.648161+0000","last_undegraded":"2024-09-10T18:17:58.648161+0000","last_fullsized":"2024-09-10T18:17:58.648161+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:42.616685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962337+0000","last_change":"2024-09-10T18:17:47.915001+0000","last_active":"2024-09-10T18:17:58.962337+0000","last_peered":"2024-09-10T18:17:58.962337+0000","last_clean":"2024-09-10T18:17:58.962337+0000","last_became_active":"2024-09-10T18:17:47.914721+0000","last_became_peered":"2024-09-10T18:17:47.914721+0000","last_unstale":"2024-09-10T18:17:58.962337+0000","last_undegraded":"2024-09-10T18:17:58.962337+0000","last_fullsized":"2024-09-10T18:17:58.962337+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:18.364951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648402+0000","last_change":"2024-09-10T18:17:51.940619+0000","last_active":"2024-09-10T18:17:58.648402+0000","last_peered":"2024-09-10T18:17:58.648402+0000","last_clean":"2024-09-10T18:17:58.648402+0000","last_became_active":"2024-09-10T18:17:51.940359+0000","last_became_peered":"2024-09-10T18:17:51.940359+0000","last_unstale":"2024-09-10T18:17:58.648402+0000","last_undegraded":"2024-09-10T18:17:58.648402+0000","last_fullsized":"2024-09-10T18:17:58.648402+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:39.062953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648418+0000","last_change":"2024-09-10T18:17:53.952413+0000","last_active":"2024-09-10T18:17:58.648418+0000","last_peered":"2024-09-10T18:17:58.648418+0000","last_clean":"2024-09-10T18:17:58.648418+0000","last_became_active":"2024-09-10T18:17:53.951759+0000","last_became_peered":"2024-09-10T18:17:53.951759+0000","last_unstale":"2024-09-10T18:17:58.648418+0000","last_undegraded":"2024-09-10T18:17:58.648418+0000","last_fullsized":"2024-09-10T18:17:58.648418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:43.378170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647909+0000","last_change":"2024-09-10T18:17:49.927821+0000","last_active":"2024-09-10T18:17:58.647909+0000","last_peered":"2024-09-10T18:17:58.647909+0000","last_clean":"2024-09-10T18:17:58.647909+0000","last_became_active":"2024-09-10T18:17:49.927525+0000","last_became_peered":"2024-09-10T18:17:49.927525+0000","last_unstale":"2024-09-10T18:17:58.647909+0000","last_undegraded":"2024-09-10T18:17:58.647909+0000","last_fullsized":"2024-09-10T18:17:58.647909+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:49.385183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686592+0000","last_change":"2024-09-10T18:17:47.913415+0000","last_active":"2024-09-10T18:17:58.686592+0000","last_peered":"2024-09-10T18:17:58.686592+0000","last_clean":"2024-09-10T18:17:58.686592+0000","last_became_active":"2024-09-10T18:17:47.913157+0000","last_became_peered":"2024-09-10T18:17:47.913157+0000","last_unstale":"2024-09-10T18:17:58.686592+0000","last_undegraded":"2024-09-10T18:17:58.686592+0000","last_fullsized":"2024-09-10T18:17:58.686592+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:15.019106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107731+0000","last_change":"2024-09-10T18:17:51.939593+0000","last_active":"2024-09-10T18:19:01.107731+0000","last_peered":"2024-09-10T18:19:01.107731+0000","last_clean":"2024-09-10T18:19:01.107731+0000","last_became_active":"2024-09-10T18:17:51.939244+0000","last_became_peered":"2024-09-10T18:17:51.939244+0000","last_unstale":"2024-09-10T18:19:01.107731+0000","last_undegraded":"2024-09-10T18:19:01.107731+0000","last_fullsized":"2024-09-10T18:19:01.107731+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:15.630619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648106+0000","last_change":"2024-09-10T18:17:53.947479+0000","last_active":"2024-09-10T18:17:58.648106+0000","last_peered":"2024-09-10T18:17:58.648106+0000","last_clean":"2024-09-10T18:17:58.648106+0000","last_became_active":"2024-09-10T18:17:53.947093+0000","last_became_peered":"2024-09-10T18:17:53.947093+0000","last_unstale":"2024-09-10T18:17:58.648106+0000","last_undegraded":"2024-09-10T18:17:58.648106+0000","last_fullsized":"2024-09-10T18:17:58.648106+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:23.472424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678409+0000","last_change":"2024-09-10T18:17:49.922842+0000","last_active":"2024-09-10T18:17:58.678409+0000","last_peered":"2024-09-10T18:17:58.678409+0000","last_clean":"2024-09-10T18:17:58.678409+0000","last_became_active":"2024-09-10T18:17:49.922246+0000","last_became_peered":"2024-09-10T18:17:49.922246+0000","last_unstale":"2024-09-10T18:17:58.678409+0000","last_undegraded":"2024-09-10T18:17:58.678409+0000","last_fullsized":"2024-09-10T18:17:58.678409+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:22.929374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961887+0000","last_change":"2024-09-10T18:17:47.912560+0000","last_active":"2024-09-10T18:17:58.961887+0000","last_peered":"2024-09-10T18:17:58.961887+0000","last_clean":"2024-09-10T18:17:58.961887+0000","last_became_active":"2024-09-10T18:17:47.912269+0000","last_became_peered":"2024-09-10T18:17:47.912269+0000","last_unstale":"2024-09-10T18:17:58.961887+0000","last_undegraded":"2024-09-10T18:17:58.961887+0000","last_fullsized":"2024-09-10T18:17:58.961887+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:28.461851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647954+0000","last_change":"2024-09-10T18:17:51.940710+0000","last_active":"2024-09-10T18:17:58.647954+0000","last_peered":"2024-09-10T18:17:58.647954+0000","last_clean":"2024-09-10T18:17:58.647954+0000","last_became_active":"2024-09-10T18:17:51.940421+0000","last_became_peered":"2024-09-10T18:17:51.940421+0000","last_unstale":"2024-09-10T18:17:58.647954+0000","last_undegraded":"2024-09-10T18:17:58.647954+0000","last_fullsized":"2024-09-10T18:17:58.647954+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:29.833364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678016+0000","last_change":"2024-09-10T18:17:53.953547+0000","last_active":"2024-09-10T18:17:58.678016+0000","last_peered":"2024-09-10T18:17:58.678016+0000","last_clean":"2024-09-10T18:17:58.678016+0000","last_became_active":"2024-09-10T18:17:53.952997+0000","last_became_peered":"2024-09-10T18:17:53.952997+0000","last_unstale":"2024-09-10T18:17:58.678016+0000","last_undegraded":"2024-09-10T18:17:58.678016+0000","last_fullsized":"2024-09-10T18:17:58.678016+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:08.141228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962236+0000","last_change":"2024-09-10T18:17:50.099303+0000","last_active":"2024-09-10T18:17:58.962236+0000","last_peered":"2024-09-10T18:17:58.962236+0000","last_clean":"2024-09-10T18:17:58.962236+0000","last_became_active":"2024-09-10T18:17:50.098970+0000","last_became_peered":"2024-09-10T18:17:50.098970+0000","last_unstale":"2024-09-10T18:17:58.962236+0000","last_undegraded":"2024-09-10T18:17:58.962236+0000","last_fullsized":"2024-09-10T18:17:58.962236+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:59.689308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.963108+0000","last_change":"2024-09-10T18:17:47.916889+0000","last_active":"2024-09-10T18:17:58.963108+0000","last_peered":"2024-09-10T18:17:58.963108+0000","last_clean":"2024-09-10T18:17:58.963108+0000","last_became_active":"2024-09-10T18:17:47.915651+0000","last_became_peered":"2024-09-10T18:17:47.915651+0000","last_unstale":"2024-09-10T18:17:58.963108+0000","last_undegraded":"2024-09-10T18:17:58.963108+0000","last_fullsized":"2024-09-10T18:17:58.963108+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:50.007519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647573+0000","last_change":"2024-09-10T18:17:51.938964+0000","last_active":"2024-09-10T18:17:58.647573+0000","last_peered":"2024-09-10T18:17:58.647573+0000","last_clean":"2024-09-10T18:17:58.647573+0000","last_became_active":"2024-09-10T18:17:51.938610+0000","last_became_peered":"2024-09-10T18:17:51.938610+0000","last_unstale":"2024-09-10T18:17:58.647573+0000","last_undegraded":"2024-09-10T18:17:58.647573+0000","last_fullsized":"2024-09-10T18:17:58.647573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:33.700812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962832+0000","last_change":"2024-09-10T18:17:53.948301+0000","last_active":"2024-09-10T18:17:58.962832+0000","last_peered":"2024-09-10T18:17:58.962832+0000","last_clean":"2024-09-10T18:17:58.962832+0000","last_became_active":"2024-09-10T18:17:53.947535+0000","last_became_peered":"2024-09-10T18:17:53.947535+0000","last_unstale":"2024-09-10T18:17:58.962832+0000","last_undegraded":"2024-09-10T18:17:58.962832+0000","last_fullsized":"2024-09-10T18:17:58.962832+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:57.956421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647457+0000","last_change":"2024-09-10T18:17:49.927734+0000","last_active":"2024-09-10T18:17:58.647457+0000","last_peered":"2024-09-10T18:17:58.647457+0000","last_clean":"2024-09-10T18:17:58.647457+0000","last_became_active":"2024-09-10T18:17:49.927418+0000","last_became_peered":"2024-09-10T18:17:49.927418+0000","last_unstale":"2024-09-10T18:17:58.647457+0000","last_undegraded":"2024-09-10T18:17:58.647457+0000","last_fullsized":"2024-09-10T18:17:58.647457+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:22.287667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.963100+0000","last_change":"2024-09-10T18:17:47.917389+0000","last_active":"2024-09-10T18:17:58.963100+0000","last_peered":"2024-09-10T18:17:58.963100+0000","last_clean":"2024-09-10T18:17:58.963100+0000","last_became_active":"2024-09-10T18:17:47.917108+0000","last_became_peered":"2024-09-10T18:17:47.917108+0000","last_unstale":"2024-09-10T18:17:58.963100+0000","last_undegraded":"2024-09-10T18:17:58.963100+0000","last_fullsized":"2024-09-10T18:17:58.963100+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:38.381474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647613+0000","last_change":"2024-09-10T18:17:51.937905+0000","last_active":"2024-09-10T18:17:58.647613+0000","last_peered":"2024-09-10T18:17:58.647613+0000","last_clean":"2024-09-10T18:17:58.647613+0000","last_became_active":"2024-09-10T18:17:51.937565+0000","last_became_peered":"2024-09-10T18:17:51.937565+0000","last_unstale":"2024-09-10T18:17:58.647613+0000","last_undegraded":"2024-09-10T18:17:58.647613+0000","last_fullsized":"2024-09-10T18:17:58.647613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:49.203573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686372+0000","last_change":"2024-09-10T18:17:53.947136+0000","last_active":"2024-09-10T18:17:58.686372+0000","last_peered":"2024-09-10T18:17:58.686372+0000","last_clean":"2024-09-10T18:17:58.686372+0000","last_became_active":"2024-09-10T18:17:53.946747+0000","last_became_peered":"2024-09-10T18:17:53.946747+0000","last_unstale":"2024-09-10T18:17:58.686372+0000","last_undegraded":"2024-09-10T18:17:58.686372+0000","last_fullsized":"2024-09-10T18:17:58.686372+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:18.722442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961136+0000","last_change":"2024-09-10T18:17:50.100732+0000","last_active":"2024-09-10T18:17:58.961136+0000","last_peered":"2024-09-10T18:17:58.961136+0000","last_clean":"2024-09-10T18:17:58.961136+0000","last_became_active":"2024-09-10T18:17:50.099740+0000","last_became_peered":"2024-09-10T18:17:50.099740+0000","last_unstale":"2024-09-10T18:17:58.961136+0000","last_undegraded":"2024-09-10T18:17:58.961136+0000","last_fullsized":"2024-09-10T18:17:58.961136+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:11.588736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"52'2","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962675+0000","last_change":"2024-09-10T18:17:47.914310+0000","last_active":"2024-09-10T18:17:58.962675+0000","last_peered":"2024-09-10T18:17:58.962675+0000","last_clean":"2024-09-10T18:17:58.962675+0000","last_became_active":"2024-09-10T18:17:47.914072+0000","last_became_peered":"2024-09-10T18:17:47.914072+0000","last_unstale":"2024-09-10T18:17:58.962675+0000","last_undegraded":"2024-09-10T18:17:58.962675+0000","last_fullsized":"2024-09-10T18:17:58.962675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:00.665604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107790+0000","last_change":"2024-09-10T18:17:51.943130+0000","last_active":"2024-09-10T18:19:01.107790+0000","last_peered":"2024-09-10T18:19:01.107790+0000","last_clean":"2024-09-10T18:19:01.107790+0000","last_became_active":"2024-09-10T18:17:51.942774+0000","last_became_peered":"2024-09-10T18:17:51.942774+0000","last_unstale":"2024-09-10T18:19:01.107790+0000","last_undegraded":"2024-09-10T18:19:01.107790+0000","last_fullsized":"2024-09-10T18:19:01.107790+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:16.246552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961478+0000","last_change":"2024-09-10T18:17:54.186495+0000","last_active":"2024-09-10T18:17:58.961478+0000","last_peered":"2024-09-10T18:17:58.961478+0000","last_clean":"2024-09-10T18:17:58.961478+0000","last_became_active":"2024-09-10T18:17:54.186225+0000","last_became_peered":"2024-09-10T18:17:54.186225+0000","last_unstale":"2024-09-10T18:17:58.961478+0000","last_undegraded":"2024-09-10T18:17:58.961478+0000","last_fullsized":"2024-09-10T18:17:58.961478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:42.174818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962001+0000","last_change":"2024-09-10T18:17:47.917528+0000","last_active":"2024-09-10T18:17:58.962001+0000","last_peered":"2024-09-10T18:17:58.962001+0000","last_clean":"2024-09-10T18:17:58.962001+0000","last_became_active":"2024-09-10T18:17:47.917270+0000","last_became_peered":"2024-09-10T18:17:47.917270+0000","last_unstale":"2024-09-10T18:17:58.962001+0000","last_undegraded":"2024-09-10T18:17:58.962001+0000","last_fullsized":"2024-09-10T18:17:58.962001+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:41.867811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686953+0000","last_change":"2024-09-10T18:17:50.099156+0000","last_active":"2024-09-10T18:17:58.686953+0000","last_peered":"2024-09-10T18:17:58.686953+0000","last_clean":"2024-09-10T18:17:58.686953+0000","last_became_active":"2024-09-10T18:17:50.098873+0000","last_became_peered":"2024-09-10T18:17:50.098873+0000","last_unstale":"2024-09-10T18:17:58.686953+0000","last_undegraded":"2024-09-10T18:17:58.686953+0000","last_fullsized":"2024-09-10T18:17:58.686953+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:20.247117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962134+0000","last_change":"2024-09-10T18:17:51.942265+0000","last_active":"2024-09-10T18:17:58.962134+0000","last_peered":"2024-09-10T18:17:58.962134+0000","last_clean":"2024-09-10T18:17:58.962134+0000","last_became_active":"2024-09-10T18:17:51.941957+0000","last_became_peered":"2024-09-10T18:17:51.941957+0000","last_unstale":"2024-09-10T18:17:58.962134+0000","last_undegraded":"2024-09-10T18:17:58.962134+0000","last_fullsized":"2024-09-10T18:17:58.962134+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:49.780993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962896+0000","last_change":"2024-09-10T18:17:54.189989+0000","last_active":"2024-09-10T18:17:58.962896+0000","last_peered":"2024-09-10T18:17:58.962896+0000","last_clean":"2024-09-10T18:17:58.962896+0000","last_became_active":"2024-09-10T18:17:54.189643+0000","last_became_peered":"2024-09-10T18:17:54.189643+0000","last_unstale":"2024-09-10T18:17:58.962896+0000","last_undegraded":"2024-09-10T18:17:58.962896+0000","last_fullsized":"2024-09-10T18:17:58.962896+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:54.564734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678561+0000","last_change":"2024-09-10T18:17:50.100249+0000","last_active":"2024-09-10T18:17:58.678561+0000","last_peered":"2024-09-10T18:17:58.678561+0000","last_clean":"2024-09-10T18:17:58.678561+0000","last_became_active":"2024-09-10T18:17:50.099961+0000","last_became_peered":"2024-09-10T18:17:50.099961+0000","last_unstale":"2024-09-10T18:17:58.678561+0000","last_undegraded":"2024-09-10T18:17:58.678561+0000","last_fullsized":"2024-09-10T18:17:58.678561+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:09.454909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962766+0000","last_change":"2024-09-10T18:17:47.915387+0000","last_active":"2024-09-10T18:17:58.962766+0000","last_peered":"2024-09-10T18:17:58.962766+0000","last_clean":"2024-09-10T18:17:58.962766+0000","last_became_active":"2024-09-10T18:17:47.915143+0000","last_became_peered":"2024-09-10T18:17:47.915143+0000","last_unstale":"2024-09-10T18:17:58.962766+0000","last_undegraded":"2024-09-10T18:17:58.962766+0000","last_fullsized":"2024-09-10T18:17:58.962766+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:47.751991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686891+0000","last_change":"2024-09-10T18:17:51.943838+0000","last_active":"2024-09-10T18:17:58.686891+0000","last_peered":"2024-09-10T18:17:58.686891+0000","last_clean":"2024-09-10T18:17:58.686891+0000","last_became_active":"2024-09-10T18:17:51.943532+0000","last_became_peered":"2024-09-10T18:17:51.943532+0000","last_unstale":"2024-09-10T18:17:58.686891+0000","last_undegraded":"2024-09-10T18:17:58.686891+0000","last_fullsized":"2024-09-10T18:17:58.686891+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:05.292320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648759+0000","last_change":"2024-09-10T18:17:53.951450+0000","last_active":"2024-09-10T18:17:58.648759+0000","last_peered":"2024-09-10T18:17:58.648759+0000","last_clean":"2024-09-10T18:17:58.648759+0000","last_became_active":"2024-09-10T18:17:53.951185+0000","last_became_peered":"2024-09-10T18:17:53.951185+0000","last_unstale":"2024-09-10T18:17:58.648759+0000","last_undegraded":"2024-09-10T18:17:58.648759+0000","last_fullsized":"2024-09-10T18:17:58.648759+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:06.420044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961590+0000","last_change":"2024-09-10T18:17:50.099290+0000","last_active":"2024-09-10T18:17:58.961590+0000","last_peered":"2024-09-10T18:17:58.961590+0000","last_clean":"2024-09-10T18:17:58.961590+0000","last_became_active":"2024-09-10T18:17:50.099015+0000","last_became_peered":"2024-09-10T18:17:50.099015+0000","last_unstale":"2024-09-10T18:17:58.961590+0000","last_undegraded":"2024-09-10T18:17:58.961590+0000","last_fullsized":"2024-09-10T18:17:58.961590+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:15.354709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678185+0000","last_change":"2024-09-10T18:17:47.911957+0000","last_active":"2024-09-10T18:17:58.678185+0000","last_peered":"2024-09-10T18:17:58.678185+0000","last_clean":"2024-09-10T18:17:58.678185+0000","last_became_active":"2024-09-10T18:17:47.911575+0000","last_became_peered":"2024-09-10T18:17:47.911575+0000","last_unstale":"2024-09-10T18:17:58.678185+0000","last_undegraded":"2024-09-10T18:17:58.678185+0000","last_fullsized":"2024-09-10T18:17:58.678185+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:28.065372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107864+0000","last_change":"2024-09-10T18:17:51.942856+0000","last_active":"2024-09-10T18:19:01.107864+0000","last_peered":"2024-09-10T18:19:01.107864+0000","last_clean":"2024-09-10T18:19:01.107864+0000","last_became_active":"2024-09-10T18:17:51.941697+0000","last_became_peered":"2024-09-10T18:17:51.941697+0000","last_unstale":"2024-09-10T18:19:01.107864+0000","last_undegraded":"2024-09-10T18:19:01.107864+0000","last_fullsized":"2024-09-10T18:19:01.107864+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:12.669121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648115+0000","last_change":"2024-09-10T18:17:53.948708+0000","last_active":"2024-09-10T18:17:58.648115+0000","last_peered":"2024-09-10T18:17:58.648115+0000","last_clean":"2024-09-10T18:17:58.648115+0000","last_became_active":"2024-09-10T18:17:53.948404+0000","last_became_peered":"2024-09-10T18:17:53.948404+0000","last_unstale":"2024-09-10T18:17:58.648115+0000","last_undegraded":"2024-09-10T18:17:58.648115+0000","last_fullsized":"2024-09-10T18:17:58.648115+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:14.775697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686909+0000","last_change":"2024-09-10T18:17:50.098867+0000","last_active":"2024-09-10T18:17:58.686909+0000","last_peered":"2024-09-10T18:17:58.686909+0000","last_clean":"2024-09-10T18:17:58.686909+0000","last_became_active":"2024-09-10T18:17:50.098593+0000","last_became_peered":"2024-09-10T18:17:50.098593+0000","last_unstale":"2024-09-10T18:17:58.686909+0000","last_undegraded":"2024-09-10T18:17:58.686909+0000","last_fullsized":"2024-09-10T18:17:58.686909+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:17.229907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961997+0000","last_change":"2024-09-10T18:17:47.913498+0000","last_active":"2024-09-10T18:17:58.961997+0000","last_peered":"2024-09-10T18:17:58.961997+0000","last_clean":"2024-09-10T18:17:58.961997+0000","last_became_active":"2024-09-10T18:17:47.913060+0000","last_became_peered":"2024-09-10T18:17:47.913060+0000","last_unstale":"2024-09-10T18:17:58.961997+0000","last_undegraded":"2024-09-10T18:17:58.961997+0000","last_fullsized":"2024-09-10T18:17:58.961997+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:34:19.558122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:19:01.107613+0000","last_change":"2024-09-10T18:17:51.941630+0000","last_active":"2024-09-10T18:19:01.107613+0000","last_peered":"2024-09-10T18:19:01.107613+0000","last_clean":"2024-09-10T18:19:01.107613+0000","last_became_active":"2024-09-10T18:17:51.940189+0000","last_became_peered":"2024-09-10T18:17:51.940189+0000","last_unstale":"2024-09-10T18:19:01.107613+0000","last_undegraded":"2024-09-10T18:19:01.107613+0000","last_fullsized":"2024-09-10T18:19:01.107613+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:44.342180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678332+0000","last_change":"2024-09-10T18:17:53.956251+0000","last_active":"2024-09-10T18:17:58.678332+0000","last_peered":"2024-09-10T18:17:58.678332+0000","last_clean":"2024-09-10T18:17:58.678332+0000","last_became_active":"2024-09-10T18:17:53.955973+0000","last_became_peered":"2024-09-10T18:17:53.955973+0000","last_unstale":"2024-09-10T18:17:58.678332+0000","last_undegraded":"2024-09-10T18:17:58.678332+0000","last_fullsized":"2024-09-10T18:17:58.678332+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:25.486164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647785+0000","last_change":"2024-09-10T18:17:49.922415+0000","last_active":"2024-09-10T18:17:58.647785+0000","last_peered":"2024-09-10T18:17:58.647785+0000","last_clean":"2024-09-10T18:17:58.647785+0000","last_became_active":"2024-09-10T18:17:49.922127+0000","last_became_peered":"2024-09-10T18:17:49.922127+0000","last_unstale":"2024-09-10T18:17:58.647785+0000","last_undegraded":"2024-09-10T18:17:58.647785+0000","last_fullsized":"2024-09-10T18:17:58.647785+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:18.842298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648033+0000","last_change":"2024-09-10T18:17:47.916076+0000","last_active":"2024-09-10T18:17:58.648033+0000","last_peered":"2024-09-10T18:17:58.648033+0000","last_clean":"2024-09-10T18:17:58.648033+0000","last_became_active":"2024-09-10T18:17:47.915311+0000","last_became_peered":"2024-09-10T18:17:47.915311+0000","last_unstale":"2024-09-10T18:17:58.648033+0000","last_undegraded":"2024-09-10T18:17:58.648033+0000","last_fullsized":"2024-09-10T18:17:58.648033+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:36.286026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.648827+0000","last_change":"2024-09-10T18:17:51.938586+0000","last_active":"2024-09-10T18:17:58.648827+0000","last_peered":"2024-09-10T18:17:58.648827+0000","last_clean":"2024-09-10T18:17:58.648827+0000","last_became_active":"2024-09-10T18:17:51.938324+0000","last_became_peered":"2024-09-10T18:17:51.938324+0000","last_unstale":"2024-09-10T18:17:58.648827+0000","last_undegraded":"2024-09-10T18:17:58.648827+0000","last_fullsized":"2024-09-10T18:17:58.648827+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:48.130738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686128+0000","last_change":"2024-09-10T18:17:53.954175+0000","last_active":"2024-09-10T18:17:58.686128+0000","last_peered":"2024-09-10T18:17:58.686128+0000","last_clean":"2024-09-10T18:17:58.686128+0000","last_became_active":"2024-09-10T18:17:53.953796+0000","last_became_peered":"2024-09-10T18:17:53.953796+0000","last_unstale":"2024-09-10T18:17:58.686128+0000","last_undegraded":"2024-09-10T18:17:58.686128+0000","last_fullsized":"2024-09-10T18:17:58.686128+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:46.386483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.649104+0000","last_change":"2024-09-10T18:17:49.929377+0000","last_active":"2024-09-10T18:17:58.649104+0000","last_peered":"2024-09-10T18:17:58.649104+0000","last_clean":"2024-09-10T18:17:58.649104+0000","last_became_active":"2024-09-10T18:17:49.928833+0000","last_became_peered":"2024-09-10T18:17:49.928833+0000","last_unstale":"2024-09-10T18:17:58.649104+0000","last_undegraded":"2024-09-10T18:17:58.649104+0000","last_fullsized":"2024-09-10T18:17:58.649104+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:32.132956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.963048+0000","last_change":"2024-09-10T18:17:47.913760+0000","last_active":"2024-09-10T18:17:58.963048+0000","last_peered":"2024-09-10T18:17:58.963048+0000","last_clean":"2024-09-10T18:17:58.963048+0000","last_became_active":"2024-09-10T18:17:47.913426+0000","last_became_peered":"2024-09-10T18:17:47.913426+0000","last_unstale":"2024-09-10T18:17:58.963048+0000","last_undegraded":"2024-09-10T18:17:58.963048+0000","last_fullsized":"2024-09-10T18:17:58.963048+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:43.518962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686503+0000","last_change":"2024-09-10T18:17:51.938886+0000","last_active":"2024-09-10T18:17:58.686503+0000","last_peered":"2024-09-10T18:17:58.686503+0000","last_clean":"2024-09-10T18:17:58.686503+0000","last_became_active":"2024-09-10T18:17:51.938591+0000","last_became_peered":"2024-09-10T18:17:51.938591+0000","last_unstale":"2024-09-10T18:17:58.686503+0000","last_undegraded":"2024-09-10T18:17:58.686503+0000","last_fullsized":"2024-09-10T18:17:58.686503+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:24.065760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.677845+0000","last_change":"2024-09-10T18:17:53.956943+0000","last_active":"2024-09-10T18:17:58.677845+0000","last_peered":"2024-09-10T18:17:58.677845+0000","last_clean":"2024-09-10T18:17:58.677845+0000","last_became_active":"2024-09-10T18:17:53.956663+0000","last_became_peered":"2024-09-10T18:17:53.956663+0000","last_unstale":"2024-09-10T18:17:58.677845+0000","last_undegraded":"2024-09-10T18:17:58.677845+0000","last_fullsized":"2024-09-10T18:17:58.677845+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:43.143114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686272+0000","last_change":"2024-09-10T18:17:50.099052+0000","last_active":"2024-09-10T18:17:58.686272+0000","last_peered":"2024-09-10T18:17:58.686272+0000","last_clean":"2024-09-10T18:17:58.686272+0000","last_became_active":"2024-09-10T18:17:50.098700+0000","last_became_peered":"2024-09-10T18:17:50.098700+0000","last_unstale":"2024-09-10T18:17:58.686272+0000","last_undegraded":"2024-09-10T18:17:58.686272+0000","last_fullsized":"2024-09-10T18:17:58.686272+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:57.595169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962204+0000","last_change":"2024-09-10T18:17:47.917189+0000","last_active":"2024-09-10T18:17:58.962204+0000","last_peered":"2024-09-10T18:17:58.962204+0000","last_clean":"2024-09-10T18:17:58.962204+0000","last_became_active":"2024-09-10T18:17:47.916930+0000","last_became_peered":"2024-09-10T18:17:47.916930+0000","last_unstale":"2024-09-10T18:17:58.962204+0000","last_undegraded":"2024-09-10T18:17:58.962204+0000","last_fullsized":"2024-09-10T18:17:58.962204+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:09.259941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961072+0000","last_change":"2024-09-10T18:17:51.944246+0000","last_active":"2024-09-10T18:17:58.961072+0000","last_peered":"2024-09-10T18:17:58.961072+0000","last_clean":"2024-09-10T18:17:58.961072+0000","last_became_active":"2024-09-10T18:17:51.943973+0000","last_became_peered":"2024-09-10T18:17:51.943973+0000","last_unstale":"2024-09-10T18:17:58.961072+0000","last_undegraded":"2024-09-10T18:17:58.961072+0000","last_fullsized":"2024-09-10T18:17:58.961072+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:23.600285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686577+0000","last_change":"2024-09-10T18:17:54.186992+0000","last_active":"2024-09-10T18:17:58.686577+0000","last_peered":"2024-09-10T18:17:58.686577+0000","last_clean":"2024-09-10T18:17:58.686577+0000","last_became_active":"2024-09-10T18:17:54.186675+0000","last_became_peered":"2024-09-10T18:17:54.186675+0000","last_unstale":"2024-09-10T18:17:58.686577+0000","last_undegraded":"2024-09-10T18:17:58.686577+0000","last_fullsized":"2024-09-10T18:17:58.686577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:19.450768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686123+0000","last_change":"2024-09-10T18:17:49.929557+0000","last_active":"2024-09-10T18:17:58.686123+0000","last_peered":"2024-09-10T18:17:58.686123+0000","last_clean":"2024-09-10T18:17:58.686123+0000","last_became_active":"2024-09-10T18:17:49.928925+0000","last_became_peered":"2024-09-10T18:17:49.928925+0000","last_unstale":"2024-09-10T18:17:58.686123+0000","last_undegraded":"2024-09-10T18:17:58.686123+0000","last_fullsized":"2024-09-10T18:17:58.686123+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:35.327522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678581+0000","last_change":"2024-09-10T18:17:47.914095+0000","last_active":"2024-09-10T18:17:58.678581+0000","last_peered":"2024-09-10T18:17:58.678581+0000","last_clean":"2024-09-10T18:17:58.678581+0000","last_became_active":"2024-09-10T18:17:47.913545+0000","last_became_peered":"2024-09-10T18:17:47.913545+0000","last_unstale":"2024-09-10T18:17:58.678581+0000","last_undegraded":"2024-09-10T18:17:58.678581+0000","last_fullsized":"2024-09-10T18:17:58.678581+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:56.279468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.962303+0000","last_change":"2024-09-10T18:17:51.946475+0000","last_active":"2024-09-10T18:17:58.962303+0000","last_peered":"2024-09-10T18:17:58.962303+0000","last_clean":"2024-09-10T18:17:58.962303+0000","last_became_active":"2024-09-10T18:17:51.946154+0000","last_became_peered":"2024-09-10T18:17:51.946154+0000","last_unstale":"2024-09-10T18:17:58.962303+0000","last_undegraded":"2024-09-10T18:17:58.962303+0000","last_fullsized":"2024-09-10T18:17:58.962303+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:51.519862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961423+0000","last_change":"2024-09-10T18:17:53.948163+0000","last_active":"2024-09-10T18:17:58.961423+0000","last_peered":"2024-09-10T18:17:58.961423+0000","last_clean":"2024-09-10T18:17:58.961423+0000","last_became_active":"2024-09-10T18:17:53.947578+0000","last_became_peered":"2024-09-10T18:17:53.947578+0000","last_unstale":"2024-09-10T18:17:58.961423+0000","last_undegraded":"2024-09-10T18:17:58.961423+0000","last_fullsized":"2024-09-10T18:17:58.961423+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:31.078803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.647849+0000","last_change":"2024-09-10T18:17:53.953655+0000","last_active":"2024-09-10T18:17:58.647849+0000","last_peered":"2024-09-10T18:17:58.647849+0000","last_clean":"2024-09-10T18:17:58.647849+0000","last_became_active":"2024-09-10T18:17:53.953374+0000","last_became_peered":"2024-09-10T18:17:53.953374+0000","last_unstale":"2024-09-10T18:17:58.647849+0000","last_undegraded":"2024-09-10T18:17:58.647849+0000","last_fullsized":"2024-09-10T18:17:58.647849+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:52.924478+0000","last_clean_scrub_stamp":"2024-09-10T18:17:52.924478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:44.017375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.686219+0000","last_change":"2024-09-10T18:17:47.917657+0000","last_active":"2024-09-10T18:17:58.686219+0000","last_peered":"2024-09-10T18:17:58.686219+0000","last_clean":"2024-09-10T18:17:58.686219+0000","last_became_active":"2024-09-10T18:17:47.917323+0000","last_became_peered":"2024-09-10T18:17:47.917323+0000","last_unstale":"2024-09-10T18:17:58.686219+0000","last_undegraded":"2024-09-10T18:17:58.686219+0000","last_fullsized":"2024-09-10T18:17:58.686219+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:46.890276+0000","last_clean_scrub_stamp":"2024-09-10T18:17:46.890276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:11.321171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.961593+0000","last_change":"2024-09-10T18:17:49.925378+0000","last_active":"2024-09-10T18:17:58.961593+0000","last_peered":"2024-09-10T18:17:58.961593+0000","last_clean":"2024-09-10T18:17:58.961593+0000","last_became_active":"2024-09-10T18:17:49.925089+0000","last_became_peered":"2024-09-10T18:17:49.925089+0000","last_unstale":"2024-09-10T18:17:58.961593+0000","last_undegraded":"2024-09-10T18:17:58.961593+0000","last_fullsized":"2024-09-10T18:17:58.961593+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:48.900158+0000","last_clean_scrub_stamp":"2024-09-10T18:17:48.900158+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:26.777992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:17:58.678397+0000","last_change":"2024-09-10T18:17:51.942242+0000","last_active":"2024-09-10T18:17:58.678397+0000","last_peered":"2024-09-10T18:17:58.678397+0000","last_clean":"2024-09-10T18:17:58.678397+0000","last_became_active":"2024-09-10T18:17:51.941974+0000","last_became_peered":"2024-09-10T18:17:51.941974+0000","last_unstale":"2024-09-10T18:17:58.678397+0000","last_undegraded":"2024-09-10T18:17:58.678397+0000","last_fullsized":"2024-09-10T18:17:58.678397+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:17:50.911480+0000","last_clean_scrub_stamp":"2024-09-10T18:17:50.911480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:13.060700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430229,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298536,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446616,"statfs":{"total":95995035648,"available":95689334784,"internally_reserved":0,"allocated":1212416,"data_stored":823893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.47299999999999998}]},{"osd":1,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.48599999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":17.661000000000001,"5min":17.661000000000001,"15min":17.661000000000001},"last":0.46500000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.45800000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.433},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":17.603000000000002,"5min":17.603000000000002,"15min":17.603000000000002},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.41999999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.315},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.35999999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.40400000000000003}]}]},{"osd":6,"up_from":42,"seq":180388626458,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298512,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446640,"statfs":{"total":95995035648,"available":95689359360,"internally_reserved":0,"allocated":1183744,"data_stored":821351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.51500000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.624}]},{"osd":2,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.498},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.63700000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52400000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.373}]},{"osd":5,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.41399999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:18:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.44900000000000001}]}]},{"osd":5,"up_from":36,"seq":154618822687,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":231670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:18:51 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.501,"15min":0.501},"min":{"1min":0.35299999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.47499999999999998,"15min":0.47199999999999998},"min":{"1min":0.35699999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Sep 10 18:18:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.34799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.66300000000000003,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.5,"15min":0.5},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.69599999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.52000000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:18:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48799999999999999,"15min":0.48899999999999999},"min":{"1min":0.38500000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.58999999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.41999999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:18:51 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50900000000000001,"15min":0.51100000000000001},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.64900000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.39400000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.49099999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:18:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.35499999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.39100000000000001,"15min":0.376},"min":{"1min":0.28199999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.50700000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.437},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.54100000000000004}]},{"osd":7,"last update":"Tue Sep 10 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.56699999999999995}]}]},{"osd":4,"up_from":30,"seq":128849018914,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297956,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447196,"statfs":{"total":95995035648,"available":95689928704,"internally_reserved":0,"allocated":614400,"data_stored":232672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.504,"15min":0.504},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.442}]},{"osd":1,"last update":"Tue Sep 10 18:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.501},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64700000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.52100000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.504,"15min":0.503},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.496,"15min":0.497},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67800000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.38300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41499999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67200000000000004,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.47699999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.372,"15min":0.36799999999999999},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.37},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.36599999999999999,"15min":0.35999999999999999},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.35999999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.33300000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:18:34 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.34899999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182439,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297964,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447188,"statfs":{"total":95995035648,"available":95689920512,"internally_reserved":0,"allocated":622592,"data_stored":232130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:19:07 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36399999999999999,"15min":0.37},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.308},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39700000000000002,"15min":0.40799999999999997},"min":{"1min":0.25600000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.442}]},{"osd":1,"last update":"Tue Sep 10 18:19:07 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.64200000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.40799999999999997,"15min":0.41999999999999998},"min":{"1min":0.24299999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.42499999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:19:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.437,"15min":0.44700000000000001},"min":{"1min":0.26600000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.42099999999999999,"15min":0.434},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.48999999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.46200000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:18:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68700000000000006,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.371},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.52700000000000002},"min":{"1min":0.375,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:18:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.501},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39800000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":6.5990000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":6.5839999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.47799999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297936,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447216,"statfs":{"total":95995035648,"available":95689949184,"internally_reserved":0,"allocated":593920,"data_stored":232076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35399999999999998,"15min":0.35099999999999998},"min":{"1min":0.22900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44900000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.372,"15min":0.372},"min":{"1min":0.248,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.50800000000000001,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.30299999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.38800000000000001,"15min":0.39300000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.40400000000000003,"15min":0.40600000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.313,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.29199999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68000000000000005,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.443,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.72199999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.45800000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:18:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.375},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.41699999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.38600000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.432}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":233655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33400000000000002,"15min":0.31900000000000001},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.35399999999999998,"15min":0.33800000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.56899999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.22600000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35299999999999998,"15min":0.33900000000000002},"min":{"1min":0.24099999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35499999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.503,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.31}]},{"osd":3,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41099999999999998,"15min":0.41299999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.50800000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.41499999999999998,"15min":0.41599999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.27800000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72099999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.39000000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.37}]},{"osd":6,"last update":"Tue Sep 10 18:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.57299999999999995}]},{"osd":7,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.378},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.54500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738420,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298540,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446612,"statfs":{"total":95995035648,"available":95689330688,"internally_reserved":0,"allocated":1216512,"data_stored":823603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32400000000000001,"15min":0.30599999999999999},"min":{"1min":0.19400000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.436},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.34899999999999998,"15min":0.32100000000000001},"min":{"1min":0.247,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.48599999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.36399999999999999,"15min":0.36599999999999999},"min":{"1min":0.219,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.45400000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.39000000000000001,"15min":0.38300000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51600000000000001,"5min":0.753,"15min":0.753},"last":0.41299999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.249,"5min":0.247,"15min":0.247},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.496},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.46200000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:18:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69699999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68500000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Sep 10 18:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.40000000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.67100000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55500000000000005,"15min":0.55900000000000005},"min":{"1min":0.39300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.66300000000000003}]},{"osd":6,"last update":"Tue Sep 10 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71399999999999997}]},{"osd":7,"last update":"Tue Sep 10 18:18:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.623}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:19:15.084 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T18:19:15.084 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T18:19:15.084 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T18:19:15.084 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T18:19:15.328 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:19:15.696 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:15 smithi099 ceph-mon[27037]: from='client.34655 v1:172.21.15.99:0/2608175556' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:19:15.696 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:15 smithi099 ceph-mon[27037]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:15.696 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:15 smithi099 ceph-mon[37557]: from='client.34655 v1:172.21.15.99:0/2608175556' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:19:15.696 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:15 smithi099 ceph-mon[37557]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:15.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:15 smithi133 ceph-mon[31214]: from='client.34655 v1:172.21.15.99:0/2608175556' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:19:15.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:15 smithi133 ceph-mon[31214]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:17.119 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T18:19:17.119 INFO:teuthology.orchestra.run.smithi099.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T18:19:17.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:17 smithi133 ceph-mon[31214]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:17.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:17 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3490920636' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:19:17.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:19:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:19:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:19:17.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:17 smithi099 ceph-mon[27037]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:17.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:17 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3490920636' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:19:17.717 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:17 smithi099 ceph-mon[37557]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:17.717 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:17 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3490920636' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:19:17.740 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T18:19:17.741 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T18:19:17.741 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T18:19:17.756 INFO:tasks.workunit:Pulling workunits from ref fad5356a94e6044ce1062d748daa5a0f2ed53a7d 2024-09-10T18:19:17.757 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T18:19:17.757 DEBUG:teuthology.orchestra.run.smithi099:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T18:19:17.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:19:17.773 INFO:teuthology.orchestra.run.smithi099.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T18:19:17.774 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-10T18:19:17.828 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-10T18:19:17.829 DEBUG:teuthology.orchestra.run.smithi099:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-10T18:19:17.884 INFO:tasks.workunit:timeout=3h 2024-09-10T18:19:17.884 INFO:tasks.workunit:cleanup=True 2024-09-10T18:19:17.885 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/adk3798/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout fad5356a94e6044ce1062d748daa5a0f2ed53a7d 2024-09-10T18:19:17.940 INFO:tasks.workunit.client.0.smithi099.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T18:19:18.490 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:18 smithi133 bash[62342]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T18:19:19.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:19 smithi133 ceph-mon[31214]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:19.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:19 smithi099 ceph-mon[27037]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:19.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:19 smithi099 ceph-mon[37557]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:20.240 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:19 smithi133 bash[62342]: Writing manifest to image destination 2024-09-10T18:19:21.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:21 smithi133 ceph-mon[31214]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:21.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:21 smithi099 ceph-mon[27037]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:21.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:21 smithi099 ceph-mon[37557]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:22.741 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:22 smithi133 podman[62342]: 2024-09-10 18:19:22.335856604 +0000 UTC m=+8.492730361 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:19:22.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:22 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-10T18:19:22.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:22 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-10T18:19:22.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:22 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2024-09-10T18:19:22.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:22 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:19:22.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:22 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-10T18:19:22.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:22 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-10T18:19:22.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:22 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2024-09-10T18:19:22.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:22 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:19:22.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:22 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-10T18:19:22.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:22 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-10T18:19:22.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:22 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]: dispatch 2024-09-10T18:19:22.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:22 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:19:23.314 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 podman[62342]: 2024-09-10 18:19:23.001548581 +0000 UTC m=+9.158422377 container create 09d061edb509245913129ac8e697d9b1d918eccf030657153a422d28ac34300c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:19:23.314 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 podman[62342]: 2024-09-10 18:19:23.211903072 +0000 UTC m=+9.368776836 container init 09d061edb509245913129ac8e697d9b1d918eccf030657153a422d28ac34300c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:19:23.314 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 podman[62342]: 2024-09-10 18:19:23.215184079 +0000 UTC m=+9.372057832 container start 09d061edb509245913129ac8e697d9b1d918eccf030657153a422d28ac34300c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:19:23.314 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.219Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:19:23.314 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.219Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:19:23.314 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.220Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:19:23.314 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.220Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:19:23.314 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.220Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:19:23.315 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.220Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:19:23.315 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:19:23.315 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:19:23.315 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:19:23.315 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:19:23.315 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:19:23.315 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:19:23.316 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:19:23.316 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:19:23.316 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:19:23.316 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:19:23.316 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:19:23.316 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:19:23.316 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:19:23.316 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:19:23.316 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:19:23.316 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:19:23.316 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:19:23.317 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:19:23.317 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:19:23.317 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:19:23.317 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:19:23.317 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:19:23.317 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:19:23.317 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:19:23.317 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:19:23.317 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:19:23.317 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:19:23.317 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:19:23.317 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:19:23.318 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:19:23.318 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:19:23.318 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:19:23.318 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:19:23.318 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:19:23.318 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:19:23.318 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:19:23.318 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:19:23.318 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:19:23.318 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:19:23.318 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:19:23.318 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:19:23.319 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:19:23.319 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:19:23.319 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:19:23.319 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:19:23.319 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:19:23.319 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b[62418]: ts=2024-09-10T18:19:23.221Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:19:23.319 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 bash[62342]: 09d061edb509245913129ac8e697d9b1d918eccf030657153a422d28ac34300c 2024-09-10T18:19:23.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:19:23.741 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:19:23 smithi133 systemd[1]: Started Ceph node-exporter.b for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:19:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-mon[31214]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-10T18:19:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-10T18:19:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]': finished 2024-09-10T18:19:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-mon[31214]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:19:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:23.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:23 smithi099 ceph-mon[27037]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:23.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-10T18:19:23.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-10T18:19:23.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]': finished 2024-09-10T18:19:23.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:23 smithi099 ceph-mon[27037]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:19:23.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:23.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:23 smithi099 ceph-mon[37557]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:23.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-10T18:19:23.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-10T18:19:23.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]': finished 2024-09-10T18:19:23.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:23 smithi099 ceph-mon[37557]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:19:23.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:24.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:24 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:24.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:24 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:24.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:24 smithi133 ceph-mon[31214]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:19:24.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:24 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:24.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:24 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:24.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:24 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:24.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:24 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:24.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:24.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:24.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[27037]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:19:24.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:24.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:24.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:24.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:24.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:24.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:24.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[37557]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:19:24.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:24.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:24.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:24.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:24 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:25.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:25 smithi099 ceph-mon[27037]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:19:25.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:25 smithi099 ceph-mon[27037]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T18:19:25.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:25 smithi099 ceph-mon[37557]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:19:25.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:25 smithi099 ceph-mon[37557]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T18:19:25.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:25 smithi133 ceph-mon[31214]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:19:25.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:25 smithi133 ceph-mon[31214]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T18:19:27.376 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:19:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:19:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:19:27.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:27 smithi133 ceph-mon[31214]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:19:27.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:27 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:27.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:27 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:27.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:27 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:27.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:27 smithi099 ceph-mon[37557]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:19:27.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:27 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:27.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:27 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:27.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:27 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:27.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:27 smithi099 ceph-mon[27037]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:19:27.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:27 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:27.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:27 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:27.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:27 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:28.660 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:28 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:28.660 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:28 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:28.660 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:28 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:28.660 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:28 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:28.661 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:28 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:28.661 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:28 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:28.661 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:28 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:28.661 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:28 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:28.661 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:28 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:28.661 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:28 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:28.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:28 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:28.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:28 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:28.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:28 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:28.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:28 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:28.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:28 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:29.748 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:29 smithi099 ceph-mon[27037]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:19:29.748 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:29 smithi099 ceph-mon[27037]: Reconfiguring daemon alertmanager.a on smithi099 2024-09-10T18:19:29.748 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:29 smithi099 ceph-mon[27037]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-10T18:19:29.749 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:29 smithi099 ceph-mon[27037]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-10T18:19:29.749 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:29 smithi099 ceph-mon[27037]: Cluster is now healthy 2024-09-10T18:19:29.749 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:29 smithi099 ceph-mon[37557]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:19:29.749 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:29 smithi099 ceph-mon[37557]: Reconfiguring daemon alertmanager.a on smithi099 2024-09-10T18:19:29.749 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:29 smithi099 ceph-mon[37557]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-10T18:19:29.749 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:29 smithi099 ceph-mon[37557]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-10T18:19:29.749 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:29 smithi099 ceph-mon[37557]: Cluster is now healthy 2024-09-10T18:19:29.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:29 smithi133 ceph-mon[31214]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:19:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:29 smithi133 ceph-mon[31214]: Reconfiguring daemon alertmanager.a on smithi099 2024-09-10T18:19:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:29 smithi133 ceph-mon[31214]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-10T18:19:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:29 smithi133 ceph-mon[31214]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-10T18:19:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:29 smithi133 ceph-mon[31214]: Cluster is now healthy 2024-09-10T18:19:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:31 smithi099 ceph-mon[27037]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-10T18:19:31.756 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:31 smithi099 systemd[1]: Stopping Ceph alertmanager.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:19:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:31 smithi099 ceph-mon[37557]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-10T18:19:31.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:31 smithi133 ceph-mon[31214]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-10T18:19:32.131 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:31 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[70410]: ts=2024-09-10T18:19:31.803Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:19:32.131 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:31 smithi099 podman[72517]: 2024-09-10 18:19:31.824326146 +0000 UTC m=+0.140392304 container died c4e823da713ac206541c82ae0d3da3cbedf95dafaad8508c400cc912d7bc5e26 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:32.498 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:32 smithi099 podman[72517]: 2024-09-10 18:19:32.229432784 +0000 UTC m=+0.545498943 container cleanup c4e823da713ac206541c82ae0d3da3cbedf95dafaad8508c400cc912d7bc5e26 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:32.499 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:32 smithi099 bash[72517]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:19:32.755 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:32 smithi099 podman[72531]: 2024-09-10 18:19:32.497616315 +0000 UTC m=+0.669290599 container remove c4e823da713ac206541c82ae0d3da3cbedf95dafaad8508c400cc912d7bc5e26 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:32.755 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:32 smithi099 podman[72531]: 2024-09-10 18:19:32.598560591 +0000 UTC m=+0.770234882 volume remove ae7a421676935faa16514d5a938e28f3cc43c01dbfcc567c006d2116d3b94dbd 2024-09-10T18:19:32.755 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:32 smithi099 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T18:19:32.755 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:32 smithi099 systemd[1]: Stopped Ceph alertmanager.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:19:32.755 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:32 smithi099 systemd[1]: Starting Ceph alertmanager.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:19:33.255 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:32 smithi099 podman[72620]: 2024-09-10 18:19:32.843195397 +0000 UTC m=+0.017338440 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-10T18:19:33.255 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:32 smithi099 podman[72620]: 2024-09-10 18:19:32.991141923 +0000 UTC m=+0.165284962 volume create fb5c9cc23e7d3bb8d2f3398e397540f58b14af42212250549d437a51d4bef438 2024-09-10T18:19:33.255 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:33 smithi099 podman[72620]: 2024-09-10 18:19:33.08339467 +0000 UTC m=+0.257537708 container create 96422768b205d051fe741146ab884a674e0509bc5f3255f29e6375be0e101e86 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:33.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:33 smithi133 ceph-mon[31214]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 22 B/s, 1 objects/s recovering 2024-09-10T18:19:33.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:19:33 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:19:33.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:33 smithi099 ceph-mon[27037]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 22 B/s, 1 objects/s recovering 2024-09-10T18:19:33.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:33 smithi099 ceph-mon[37557]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 22 B/s, 1 objects/s recovering 2024-09-10T18:19:33.755 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:33 smithi099 podman[72620]: 2024-09-10 18:19:33.420013361 +0000 UTC m=+0.594156401 container init 96422768b205d051fe741146ab884a674e0509bc5f3255f29e6375be0e101e86 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:33.755 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:33 smithi099 podman[72620]: 2024-09-10 18:19:33.42326942 +0000 UTC m=+0.597412459 container start 96422768b205d051fe741146ab884a674e0509bc5f3255f29e6375be0e101e86 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:33.755 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:19:33.435Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T18:19:33.755 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:19:33.435Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T18:19:33.756 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:19:33.437Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.99 port=9094 2024-09-10T18:19:33.756 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:19:33.437Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T18:19:33.756 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:19:33.461Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:19:33.756 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:19:33.461Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:19:33.756 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:19:33.464Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T18:19:33.756 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:19:33.464Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T18:19:33.756 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:33 smithi099 bash[72620]: 96422768b205d051fe741146ab884a674e0509bc5f3255f29e6375be0e101e86 2024-09-10T18:19:33.756 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:33 smithi099 systemd[1]: Started Ceph alertmanager.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:19:34.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:34 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:34 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:34 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:34 smithi133 ceph-mon[31214]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:19:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:34 smithi133 ceph-mon[31214]: Reconfiguring daemon prometheus.a on smithi133 2024-09-10T18:19:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:34 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:34 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:34 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:34 smithi099 ceph-mon[27037]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:19:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:34 smithi099 ceph-mon[27037]: Reconfiguring daemon prometheus.a on smithi133 2024-09-10T18:19:35.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:34 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:34 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:34 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:34 smithi099 ceph-mon[37557]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:19:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:34 smithi099 ceph-mon[37557]: Reconfiguring daemon prometheus.a on smithi133 2024-09-10T18:19:35.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:35 smithi099 ceph-mon[27037]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s; 20 B/s, 1 objects/s recovering 2024-09-10T18:19:35.755 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:35 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:19:35.438Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000883069s 2024-09-10T18:19:35.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:35 smithi099 ceph-mon[37557]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s; 20 B/s, 1 objects/s recovering 2024-09-10T18:19:35.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:35 smithi133 ceph-mon[31214]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s; 20 B/s, 1 objects/s recovering 2024-09-10T18:19:36.741 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 systemd[1]: Stopping Ceph prometheus.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:19:37.157 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:19:36.831Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:19:37.157 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:19:36.831Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T18:19:37.157 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:19:36.831Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T18:19:37.157 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:19:36.831Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T18:19:37.157 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:19:36.831Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T18:19:37.157 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:19:36.831Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T18:19:37.157 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:19:36.831Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T18:19:37.157 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:19:36.831Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T18:19:37.157 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:19:36.831Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T18:19:37.157 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:19:36.832Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T18:19:37.158 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:19:36.832Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T18:19:37.158 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[60193]: ts=2024-09-10T18:19:36.832Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T18:19:37.158 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:36 smithi133 podman[63317]: 2024-09-10 18:19:36.841536926 +0000 UTC m=+0.123182617 container died 9f4d94f2b5ede47f2f5430cb8feeb93afc9d4e6253dfb80b956e8834fde99281 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:37.158 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 podman[63317]: 2024-09-10 18:19:37.155325913 +0000 UTC m=+0.436971588 container cleanup 9f4d94f2b5ede47f2f5430cb8feeb93afc9d4e6253dfb80b956e8834fde99281 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:37.158 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 bash[63317]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T18:19:37.490 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 podman[63329]: 2024-09-10 18:19:37.375388813 +0000 UTC m=+0.530266509 container remove 9f4d94f2b5ede47f2f5430cb8feeb93afc9d4e6253dfb80b956e8834fde99281 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:37.865 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:37 smithi133 ceph-mon[31214]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s; 18 B/s, 1 objects/s recovering 2024-09-10T18:19:37.865 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:37 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:19:37.865 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-10T18:19:37.865 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 systemd[1]: Stopped Ceph prometheus.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:19:37.865 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.029s CPU time. 2024-09-10T18:19:37.865 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 systemd[1]: Starting Ceph prometheus.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:19:37.866 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 podman[63422]: 2024-09-10 18:19:37.636823733 +0000 UTC m=+0.016571758 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-10T18:19:37.866 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 podman[63422]: 2024-09-10 18:19:37.759632368 +0000 UTC m=+0.139380382 container create 11a63b3b5e9ff44cc373221290a3efbca1a6ec8c792e97510f2c08c1cb0d378c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:38.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:37 smithi099 ceph-mon[27037]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s; 18 B/s, 1 objects/s recovering 2024-09-10T18:19:38.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:37 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:19:38.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:37 smithi099 ceph-mon[37557]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s; 18 B/s, 1 objects/s recovering 2024-09-10T18:19:38.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:37 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:19:38.241 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 podman[63422]: 2024-09-10 18:19:37.969881763 +0000 UTC m=+0.349629777 container init 11a63b3b5e9ff44cc373221290a3efbca1a6ec8c792e97510f2c08c1cb0d378c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:38.241 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 podman[63422]: 2024-09-10 18:19:37.972728339 +0000 UTC m=+0.352476362 container start 11a63b3b5e9ff44cc373221290a3efbca1a6ec8c792e97510f2c08c1cb0d378c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:19:38.241 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:37.996Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:19:38.241 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:37.996Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:19:38.241 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:37.996Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi133 (none))" 2024-09-10T18:19:38.241 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:37.996Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:19:38.242 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:37.996Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:19:38.242 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:37.998Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:19:38.242 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:37.999Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:19:38.242 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:37.999Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:19:38.242 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:37.999Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:19:38.242 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:38.001Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:19:38.242 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:38.001Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=4.992µs 2024-09-10T18:19:38.242 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:38.001Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:19:38.242 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:38.002Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-10T18:19:38.242 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:38.002Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-10T18:19:38.242 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:38.002Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=41.6µs wal_replay_duration=1.231786ms wbl_replay_duration=143ns total_replay_duration=1.354483ms 2024-09-10T18:19:38.242 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:38.003Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:19:38.243 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:38.003Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:19:38.243 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:38.003Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:19:38.243 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:38.017Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=13.622041ms db_storage=700ns remote_storage=1.107µs web_handler=367ns query_engine=646ns scrape=414.784µs scrape_sd=25.434µs notify=44.69µs notify_sd=7.79µs rules=12.897196ms tracing=5.642µs 2024-09-10T18:19:38.243 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:38.017Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:19:38.243 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:19:38.017Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:19:38.243 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 bash[63422]: 11a63b3b5e9ff44cc373221290a3efbca1a6ec8c792e97510f2c08c1cb0d378c 2024-09-10T18:19:38.243 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:19:38 smithi133 systemd[1]: Started Ceph prometheus.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:19:38.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:38 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:38] ENGINE Bus STOPPING 2024-09-10T18:19:38.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:38 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:38] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:19:38.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:38 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:38] ENGINE Bus STOPPED 2024-09-10T18:19:38.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:38 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:38] ENGINE Bus STARTING 2024-09-10T18:19:38.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:38 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:38] ENGINE Serving on http://:::9283 2024-09-10T18:19:38.506 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:38 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:38] ENGINE Bus STARTED 2024-09-10T18:19:38.506 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:38 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:38] ENGINE Bus STOPPING 2024-09-10T18:19:39.075 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.075 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.076 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:38 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:38] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:19:39.076 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:38 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:38] ENGINE Bus STOPPED 2024-09-10T18:19:39.076 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:38 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:38] ENGINE Bus STARTING 2024-09-10T18:19:39.076 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:38 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:38] ENGINE Serving on http://:::9283 2024-09-10T18:19:39.076 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:38 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:38] ENGINE Bus STARTED 2024-09-10T18:19:39.076 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:38 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:38] ENGINE Bus STOPPING 2024-09-10T18:19:39.076 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.076 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.076 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:19:39.077 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:19:39.077 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi099.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:19:39.077 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi099.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:19:39.402 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.403 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:19:39.403 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:19:39.403 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi133.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:19:39.403 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi133.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:19:39.403 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.403 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:19:39.403 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:19:39.403 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi133.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:19:39.403 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi133.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:19:39.403 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.403 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:39.403 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[37557]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s; 18 B/s, 1 objects/s recovering 2024-09-10T18:19:39.404 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:19:39.404 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:19:39.404 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi099.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:19:39.404 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi099.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:19:39.404 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.404 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:19:39.404 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:19:39.405 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi133.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:19:39.405 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi133.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:19:39.405 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.405 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:19:39.405 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:19:39.405 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi133.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:19:39.405 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi133.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:19:39.405 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.405 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:39.405 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-mon[27037]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s; 18 B/s, 1 objects/s recovering 2024-09-10T18:19:39.405 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:39] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:19:39.406 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:39] ENGINE Bus STOPPED 2024-09-10T18:19:39.406 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:39] ENGINE Bus STARTING 2024-09-10T18:19:39.486 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.486 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.486 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:19:39.486 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi099.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi099.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi133.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi133.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi133.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mon.0 v1:172.21.15.99:0/4216916156' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi133.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:39.487 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:39.488 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:39 smithi133 ceph-mon[31214]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s; 18 B/s, 1 objects/s recovering 2024-09-10T18:19:39.755 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:39] ENGINE Serving on http://:::9283 2024-09-10T18:19:39.755 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:39 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:19:39] ENGINE Bus STARTED 2024-09-10T18:19:41.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:41 smithi133 ceph-mon[31214]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:41.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:41 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:41.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:41 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:41.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:41 smithi099 ceph-mon[27037]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:41.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:41 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:41.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:41 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:41.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:41 smithi099 ceph-mon[37557]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:41.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:41 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:41.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:41 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:43.226 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:43.227 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:43.227 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:43.227 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:43.227 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:43.227 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:43.227 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[37557]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:43.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:19:43 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:19:43.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:43 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:43.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:43 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:43.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:43 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:43.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:43 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:43.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:43 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:43.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:43 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:43.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:43 smithi133 ceph-mon[31214]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:43.505 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:19:43.441Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.00387578s 2024-09-10T18:19:43.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:43.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:43.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:43.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:43.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:43.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:19:43.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-mon[27037]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:44.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:43 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:19:43] "GET /metrics HTTP/1.1" 200 219806 "" "Prometheus/2.43.0" 2024-09-10T18:19:44.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:44 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:44.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:44 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:44.585 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:44 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:45.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:45 smithi133 ceph-mon[31214]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:45.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:45 smithi099 ceph-mon[27037]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:45.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:45 smithi099 ceph-mon[37557]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:46 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:46 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:46 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:46 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:46 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:46.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:46 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:46.742 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:46 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:46.757 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:19:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:19:46.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:46 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:19:47.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:47 smithi133 ceph-mon[31214]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:47.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:19:47 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:19:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:19:47.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:47 smithi099 ceph-mon[27037]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:47.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:47 smithi099 ceph-mon[37557]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:49.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:49 smithi133 ceph-mon[31214]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:49 smithi099 ceph-mon[27037]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:49.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:49 smithi099 ceph-mon[37557]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:51.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:51 smithi099 ceph-mon[27037]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:51.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:51 smithi099 ceph-mon[37557]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:51.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:51 smithi133 ceph-mon[31214]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:52.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:52 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:19:52.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:52 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:19:52.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:52 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:19:53.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:53 smithi099 ceph-mon[27037]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:53.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:53 smithi099 ceph-mon[37557]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:53.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:53 smithi133 ceph-mon[31214]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:53.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:19:53 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:19:54.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:19:53 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:19:53] "GET /metrics HTTP/1.1" 200 219802 "" "Prometheus/2.43.0" 2024-09-10T18:19:54.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:54 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:54.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:54 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:54.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:54 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:19:55.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:55 smithi133 ceph-mon[31214]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:55.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:55 smithi099 ceph-mon[27037]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:55.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:55 smithi099 ceph-mon[37557]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:57.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:57 smithi133 ceph-mon[31214]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:57.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:19:57 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:19:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:19:57.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:57 smithi099 ceph-mon[27037]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:57.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:57 smithi099 ceph-mon[37557]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:19:59.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:19:59 smithi133 ceph-mon[31214]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:59.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:19:59 smithi099 ceph-mon[27037]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:19:59.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:19:59 smithi099 ceph-mon[37557]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:00.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:00 smithi133 ceph-mon[31214]: overall HEALTH_OK 2024-09-10T18:20:00.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:00 smithi099 ceph-mon[27037]: overall HEALTH_OK 2024-09-10T18:20:00.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:00 smithi099 ceph-mon[37557]: overall HEALTH_OK 2024-09-10T18:20:01.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:01 smithi133 ceph-mon[31214]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:20:01.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:01 smithi099 ceph-mon[27037]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:20:01.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:01 smithi099 ceph-mon[37557]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:20:02.861 INFO:tasks.workunit.client.0.smithi099.stderr:Note: switching to 'fad5356a94e6044ce1062d748daa5a0f2ed53a7d'. 2024-09-10T18:20:02.861 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-09-10T18:20:02.861 INFO:tasks.workunit.client.0.smithi099.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T18:20:02.861 INFO:tasks.workunit.client.0.smithi099.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T18:20:02.861 INFO:tasks.workunit.client.0.smithi099.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T18:20:02.862 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-09-10T18:20:02.862 INFO:tasks.workunit.client.0.smithi099.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T18:20:02.862 INFO:tasks.workunit.client.0.smithi099.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T18:20:02.862 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-09-10T18:20:02.862 INFO:tasks.workunit.client.0.smithi099.stderr: git switch -c 2024-09-10T18:20:02.862 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-09-10T18:20:02.862 INFO:tasks.workunit.client.0.smithi099.stderr:Or undo this operation with: 2024-09-10T18:20:02.862 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-09-10T18:20:02.862 INFO:tasks.workunit.client.0.smithi099.stderr: git switch - 2024-09-10T18:20:02.862 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-09-10T18:20:02.862 INFO:tasks.workunit.client.0.smithi099.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T18:20:02.862 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-09-10T18:20:02.863 INFO:tasks.workunit.client.0.smithi099.stderr:HEAD is now at fad5356a94e qa/upgrade: change pacific/octopus upgrade tests to use published image 2024-09-10T18:20:02.866 DEBUG:teuthology.orchestra.run.smithi099:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T18:20:02.895 INFO:tasks.workunit.client.0.smithi099.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T18:20:02.897 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T18:20:02.898 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T18:20:03.586 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T18:20:03.663 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T18:20:03.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:03 smithi099 ceph-mon[27037]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:20:03.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:03 smithi099 ceph-mon[37557]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:20:03.704 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T18:20:03.706 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T18:20:03.706 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T18:20:03.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:03 smithi133 ceph-mon[31214]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:20:03.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:20:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:20:03.744 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T18:20:03.747 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T18:20:03.747 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T18:20:03.804 INFO:tasks.workunit:Running workunits matching rados/test_python.sh on client.0... 2024-09-10T18:20:03.805 INFO:tasks.workunit:Running workunit rados/test_python.sh... 2024-09-10T18:20:03.805 DEBUG:teuthology.orchestra.run.smithi099:workunit test rados/test_python.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=fad5356a94e6044ce1062d748daa5a0f2ed53a7d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh 2024-09-10T18:20:03.864 INFO:tasks.workunit.client.0.smithi099.stderr:+ ceph osd pool create rbd 2024-09-10T18:20:04.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:20:03 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:20:03] "GET /metrics HTTP/1.1" 200 219802 "" "Prometheus/2.43.0" 2024-09-10T18:20:04.518 INFO:tasks.workunit.client.0.smithi099.stderr:pool 'rbd' already exists 2024-09-10T18:20:04.529 INFO:tasks.workunit.client.0.smithi099.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh 2024-09-10T18:20:04.529 INFO:tasks.workunit.client.0.smithi099.stderr:+ python3 -m pytest -v /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/../../../src/test/pybind/test_rados.py 2024-09-10T18:20:04.683 INFO:tasks.workunit.client.0.smithi099.stdout:============================= test session starts ============================== 2024-09-10T18:20:04.683 INFO:tasks.workunit.client.0.smithi099.stdout:platform linux -- Python 3.9.19, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3 2024-09-10T18:20:04.683 INFO:tasks.workunit.client.0.smithi099.stdout:cachedir: .pytest_cache 2024-09-10T18:20:04.683 INFO:tasks.workunit.client.0.smithi099.stdout:rootdir: /home/ubuntu/cephtest/clone.client.0/src/test/pybind, configfile: pytest.ini 2024-09-10T18:20:04.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:04 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:04.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:04 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/284158663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-10T18:20:04.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:04 smithi133 ceph-mon[31214]: from='client.34667 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-10T18:20:04.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:04 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:04.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:04 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/284158663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-10T18:20:04.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:04 smithi099 ceph-mon[27037]: from='client.34667 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-10T18:20:04.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:04 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:04.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:04 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/284158663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-10T18:20:04.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:04 smithi099 ceph-mon[37557]: from='client.34667 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-10T18:20:04.915 INFO:tasks.workunit.client.0.smithi099.stdout:collecting ... collected 89 items 2024-09-10T18:20:04.915 INFO:tasks.workunit.client.0.smithi099.stdout: 2024-09-10T18:20:04.924 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_rados_init_error PASSED [ 1%] 2024-09-10T18:20:04.971 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_rados_init PASSED [ 2%] 2024-09-10T18:20:04.985 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_ioctx_context_manager PASSED [ 3%] 2024-09-10T18:20:04.996 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_parse_argv PASSED [ 4%] 2024-09-10T18:20:05.002 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_parse_argv_empty_str PASSED [ 5%] 2024-09-10T18:20:05.010 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_configuring PASSED [ 6%] 2024-09-10T18:20:05.022 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_connected PASSED [ 7%] 2024-09-10T18:20:05.035 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_shutdown PASSED [ 8%] 2024-09-10T18:20:05.055 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_ping_monitor PASSED [ 10%] 2024-09-10T18:20:05.071 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_annotations PASSED [ 11%] 2024-09-10T18:20:05.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:05 smithi133 ceph-mon[31214]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:05.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:05 smithi133 ceph-mon[31214]: from='client.34667 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd"}]': finished 2024-09-10T18:20:05.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:05 smithi133 ceph-mon[31214]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:20:05.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:05 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/284158663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-10T18:20:05.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:05 smithi133 ceph-mon[31214]: from='client.34667 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-10T18:20:05.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:05 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/4208924911' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:20:05.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:05 smithi099 ceph-mon[27037]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:05.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:05 smithi099 ceph-mon[27037]: from='client.34667 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd"}]': finished 2024-09-10T18:20:05.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:05 smithi099 ceph-mon[27037]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:20:05.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:05 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/284158663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-10T18:20:05.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:05 smithi099 ceph-mon[27037]: from='client.34667 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-10T18:20:05.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:05 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/4208924911' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:20:05.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:05 smithi099 ceph-mon[37557]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:05.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:05 smithi099 ceph-mon[37557]: from='client.34667 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd"}]': finished 2024-09-10T18:20:05.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:05 smithi099 ceph-mon[37557]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:20:05.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:05 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/284158663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-10T18:20:05.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:05 smithi099 ceph-mon[37557]: from='client.34667 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-10T18:20:05.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:05 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/4208924911' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:20:06.425 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_create PASSED [ 12%] 2024-09-10T18:20:06.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:06 smithi133 ceph-mon[31214]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:20:06.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:06 smithi133 ceph-mon[31214]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:20:06.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:06 smithi099 ceph-mon[27037]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:20:06.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:06 smithi099 ceph-mon[27037]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:20:06.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:06 smithi099 ceph-mon[37557]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:20:06.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:06 smithi099 ceph-mon[37557]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:20:07.433 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:20:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:20:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:20:07.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:07 smithi133 ceph-mon[31214]: pgmap v59: 196 pgs: 64 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:20:07.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:07 smithi133 ceph-mon[31214]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:07.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:07 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:20:07.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:07 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:20:07.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:07 smithi099 ceph-mon[27037]: pgmap v59: 196 pgs: 64 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:20:07.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:07 smithi099 ceph-mon[27037]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:07.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:07 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:20:07.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:07 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:20:07.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:07 smithi099 ceph-mon[37557]: pgmap v59: 196 pgs: 64 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:20:07.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:07 smithi099 ceph-mon[37557]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:07.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:07 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:20:07.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:07 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:20:08.453 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_create_utf8 PASSED [ 13%] 2024-09-10T18:20:08.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:08 smithi133 ceph-mon[31214]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:20:08.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:08 smithi099 ceph-mon[27037]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:20:08.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:08 smithi099 ceph-mon[37557]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:20:09.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:09 smithi133 ceph-mon[31214]: pgmap v62: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:09.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:09 smithi133 ceph-mon[31214]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:20:09.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:09 smithi099 ceph-mon[27037]: pgmap v62: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:09.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:09 smithi099 ceph-mon[27037]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:20:09.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:09 smithi099 ceph-mon[37557]: pgmap v62: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:09.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:09 smithi099 ceph-mon[37557]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:20:10.474 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_pool_lookup_utf8 PASSED [ 14%] 2024-09-10T18:20:10.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:10 smithi133 ceph-mon[31214]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:20:10.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:10 smithi099 ceph-mon[27037]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:20:10.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:10 smithi099 ceph-mon[37557]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:20:11.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:11 smithi133 ceph-mon[31214]: pgmap v65: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:11.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:11 smithi133 ceph-mon[31214]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:20:11.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:11 smithi099 ceph-mon[27037]: pgmap v65: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:11.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:11 smithi099 ceph-mon[27037]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:20:11.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:11 smithi099 ceph-mon[37557]: pgmap v65: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:11.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:11 smithi099 ceph-mon[37557]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:20:12.492 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_eexist PASSED [ 15%] 2024-09-10T18:20:12.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:12 smithi133 ceph-mon[31214]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:20:12.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:12 smithi099 ceph-mon[27037]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:20:12.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:12 smithi099 ceph-mon[37557]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:20:13.734 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:20:13 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:20:13.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:13 smithi133 ceph-mon[31214]: pgmap v68: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:13.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:13 smithi133 ceph-mon[31214]: osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:20:14.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:13 smithi099 ceph-mon[27037]: pgmap v68: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:14.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:13 smithi099 ceph-mon[27037]: osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:20:14.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:20:13 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:20:13] "GET /metrics HTTP/1.1" 200 222121 "" "Prometheus/2.43.0" 2024-09-10T18:20:14.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:13 smithi099 ceph-mon[37557]: pgmap v68: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:14.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:13 smithi099 ceph-mon[37557]: osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:20:14.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:14 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:14.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:14 smithi133 ceph-mon[31214]: osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:20:14.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:14 smithi133 ceph-mon[31214]: osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:20:15.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:14 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:15.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:14 smithi099 ceph-mon[27037]: osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:20:15.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:14 smithi099 ceph-mon[27037]: osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:20:15.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:14 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:15.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:14 smithi099 ceph-mon[37557]: osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:20:15.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:14 smithi099 ceph-mon[37557]: osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:20:15.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:15 smithi133 ceph-mon[31214]: pgmap v71: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:15.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:15 smithi133 ceph-mon[31214]: osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:20:16.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:15 smithi099 ceph-mon[27037]: pgmap v71: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:16.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:15 smithi099 ceph-mon[27037]: osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:20:16.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:15 smithi099 ceph-mon[37557]: pgmap v71: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:16.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:15 smithi099 ceph-mon[37557]: osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:20:17.489 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:20:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:20:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:20:17.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:17 smithi133 ceph-mon[31214]: pgmap v74: 260 pgs: 96 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:17.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:17 smithi133 ceph-mon[31214]: osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:20:17.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:17 smithi133 ceph-mon[31214]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:17.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:17 smithi099 ceph-mon[27037]: pgmap v74: 260 pgs: 96 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:17.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:17 smithi099 ceph-mon[27037]: osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:20:17.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:17 smithi099 ceph-mon[27037]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:17.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:17 smithi099 ceph-mon[37557]: pgmap v74: 260 pgs: 96 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:17.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:17 smithi099 ceph-mon[37557]: osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:20:17.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:17 smithi099 ceph-mon[37557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:18.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:18 smithi099 ceph-mon[27037]: osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:20:18.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:18 smithi099 ceph-mon[37557]: osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:20:18.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:18 smithi133 ceph-mon[31214]: osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:20:19.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:19 smithi099 ceph-mon[27037]: pgmap v77: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:19.839 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:19 smithi099 ceph-mon[27037]: osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:20:19.839 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:19 smithi099 ceph-mon[37557]: pgmap v77: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:19.839 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:19 smithi099 ceph-mon[37557]: osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:20:19.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:19 smithi133 ceph-mon[31214]: pgmap v77: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:19.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:19 smithi133 ceph-mon[31214]: osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:20:20.523 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_list_pools PASSED [ 16%] 2024-09-10T18:20:20.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:20 smithi099 ceph-mon[27037]: osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:20:20.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:20 smithi099 ceph-mon[37557]: osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:20:20.992 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:20 smithi133 ceph-mon[31214]: osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:20:21.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:21 smithi133 ceph-mon[31214]: pgmap v80: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:21.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:21 smithi133 ceph-mon[31214]: osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:20:22.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:21 smithi099 ceph-mon[27037]: pgmap v80: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:22.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:21 smithi099 ceph-mon[27037]: osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:20:22.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:21 smithi099 ceph-mon[37557]: pgmap v80: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:22.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:21 smithi099 ceph-mon[37557]: osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:20:22.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:22 smithi133 ceph-mon[31214]: osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:20:22.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:22 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:20:23.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:22 smithi099 ceph-mon[27037]: osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:20:23.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:22 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:20:23.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:22 smithi099 ceph-mon[37557]: osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:20:23.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:22 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:20:23.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:20:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:20:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:23 smithi133 ceph-mon[31214]: pgmap v83: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:23 smithi133 ceph-mon[31214]: osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:20:23.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:23 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-09-10T18:20:24.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:20:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:20:23] "GET /metrics HTTP/1.1" 200 224651 "" "Prometheus/2.43.0" 2024-09-10T18:20:24.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:23 smithi099 ceph-mon[27037]: pgmap v83: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:24.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:23 smithi099 ceph-mon[27037]: osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:20:24.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:23 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-09-10T18:20:24.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:23 smithi099 ceph-mon[37557]: pgmap v83: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:24.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:23 smithi099 ceph-mon[37557]: osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:20:24.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:23 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-09-10T18:20:24.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:24 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:24.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:24 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]': finished 2024-09-10T18:20:24.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:24 smithi133 ceph-mon[31214]: osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:20:24.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:24 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:20:24.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:24 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:24.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:24 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:20:24.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:24 smithi133 ceph-mon[31214]: osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:20:24.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:24 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-09-10T18:20:25.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:25.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]': finished 2024-09-10T18:20:25.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[27037]: osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:20:25.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:20:25.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:25.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:20:25.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[27037]: osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:20:25.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-09-10T18:20:25.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:25.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]': finished 2024-09-10T18:20:25.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[37557]: osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:20:25.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:20:25.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:25.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:20:25.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[37557]: osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:20:25.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:24 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-09-10T18:20:25.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:25 smithi133 ceph-mon[31214]: pgmap v86: 228 pgs: 32 creating+peering, 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:25.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:25 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]': finished 2024-09-10T18:20:25.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:25 smithi133 ceph-mon[31214]: osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:20:26.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:25 smithi099 ceph-mon[27037]: pgmap v86: 228 pgs: 32 creating+peering, 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:26.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:25 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]': finished 2024-09-10T18:20:26.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:25 smithi099 ceph-mon[27037]: osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:20:26.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:25 smithi099 ceph-mon[37557]: pgmap v86: 228 pgs: 32 creating+peering, 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:26.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:25 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3468150133' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]': finished 2024-09-10T18:20:26.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:25 smithi099 ceph-mon[37557]: osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:20:27.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:20:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:20:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:20:27.544 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_get_pool_base_tier PASSED [ 17%] 2024-09-10T18:20:27.566 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_get_fsid PASSED [ 19%] 2024-09-10T18:20:27.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:27 smithi133 ceph-mon[31214]: pgmap v89: 228 pgs: 32 creating+peering, 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:27.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:27 smithi133 ceph-mon[31214]: osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:20:28.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:27 smithi099 ceph-mon[27037]: pgmap v89: 228 pgs: 32 creating+peering, 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:28.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:27 smithi099 ceph-mon[27037]: osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:20:28.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:27 smithi099 ceph-mon[37557]: pgmap v89: 228 pgs: 32 creating+peering, 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:28.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:27 smithi099 ceph-mon[37557]: osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:20:28.559 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_blocklist_add PASSED [ 20%] 2024-09-10T18:20:28.578 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_get_cluster_stats PASSED [ 21%] 2024-09-10T18:20:28.595 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_monitor_log PASSED [ 22%] 2024-09-10T18:20:28.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:28 smithi133 ceph-mon[31214]: osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:20:28.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:28 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1588040213' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-09-10T18:20:28.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:28 smithi133 ceph-mon[31214]: from='client.34772 ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-09-10T18:20:29.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:28 smithi099 ceph-mon[27037]: osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:20:29.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:28 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1588040213' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-09-10T18:20:29.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:28 smithi099 ceph-mon[27037]: from='client.34772 ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-09-10T18:20:29.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:28 smithi099 ceph-mon[37557]: osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:20:29.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:28 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1588040213' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-09-10T18:20:29.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:28 smithi099 ceph-mon[37557]: from='client.34772 ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-09-10T18:20:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:29 smithi133 ceph-mon[31214]: pgmap v92: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:29 smithi133 ceph-mon[31214]: from='client.34772 ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]': finished 2024-09-10T18:20:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:29 smithi133 ceph-mon[31214]: osdmap e88: 8 total, 8 up, 8 in 2024-09-10T18:20:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:29 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:29 smithi133 ceph-mon[31214]: osdmap e89: 8 total, 8 up, 8 in 2024-09-10T18:20:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:29 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/853741418' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:29 smithi133 ceph-mon[31214]: from='client.34790 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:30.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[37557]: pgmap v92: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:30.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[37557]: from='client.34772 ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]': finished 2024-09-10T18:20:30.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[37557]: osdmap e88: 8 total, 8 up, 8 in 2024-09-10T18:20:30.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:30.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[37557]: osdmap e89: 8 total, 8 up, 8 in 2024-09-10T18:20:30.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/853741418' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:30.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[37557]: from='client.34790 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:30.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[27037]: pgmap v92: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:30.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[27037]: from='client.34772 ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]': finished 2024-09-10T18:20:30.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[27037]: osdmap e88: 8 total, 8 up, 8 in 2024-09-10T18:20:30.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:30.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[27037]: osdmap e89: 8 total, 8 up, 8 in 2024-09-10T18:20:30.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/853741418' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:30.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:29 smithi099 ceph-mon[27037]: from='client.34790 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:30.869 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_last_version PASSED [ 23%] 2024-09-10T18:20:31.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:30 smithi133 ceph-mon[31214]: from='client.34790 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:31.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:30 smithi133 ceph-mon[31214]: osdmap e90: 8 total, 8 up, 8 in 2024-09-10T18:20:31.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:30 smithi099 ceph-mon[27037]: from='client.34790 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:31.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:30 smithi099 ceph-mon[27037]: osdmap e90: 8 total, 8 up, 8 in 2024-09-10T18:20:31.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:30 smithi099 ceph-mon[37557]: from='client.34790 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:31.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:30 smithi099 ceph-mon[37557]: osdmap e90: 8 total, 8 up, 8 in 2024-09-10T18:20:32.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:31 smithi133 ceph-mon[31214]: pgmap v96: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:20:32.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:31 smithi133 ceph-mon[31214]: osdmap e91: 8 total, 8 up, 8 in 2024-09-10T18:20:32.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:31 smithi099 ceph-mon[27037]: pgmap v96: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:20:32.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:31 smithi099 ceph-mon[27037]: osdmap e91: 8 total, 8 up, 8 in 2024-09-10T18:20:32.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:31 smithi099 ceph-mon[37557]: pgmap v96: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:20:32.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:31 smithi099 ceph-mon[37557]: osdmap e91: 8 total, 8 up, 8 in 2024-09-10T18:20:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:32 smithi133 ceph-mon[31214]: osdmap e92: 8 total, 8 up, 8 in 2024-09-10T18:20:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:32 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1945867039' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:32 smithi133 ceph-mon[31214]: from='client.34796 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:32 smithi133 ceph-mon[31214]: from='client.34796 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:33.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:32 smithi133 ceph-mon[31214]: osdmap e93: 8 total, 8 up, 8 in 2024-09-10T18:20:33.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:32 smithi099 ceph-mon[27037]: osdmap e92: 8 total, 8 up, 8 in 2024-09-10T18:20:33.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:32 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1945867039' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:33.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:32 smithi099 ceph-mon[27037]: from='client.34796 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:33.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:32 smithi099 ceph-mon[27037]: from='client.34796 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:33.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:32 smithi099 ceph-mon[27037]: osdmap e93: 8 total, 8 up, 8 in 2024-09-10T18:20:33.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:32 smithi099 ceph-mon[37557]: osdmap e92: 8 total, 8 up, 8 in 2024-09-10T18:20:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:32 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1945867039' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:32 smithi099 ceph-mon[37557]: from='client.34796 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:32 smithi099 ceph-mon[37557]: from='client.34796 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:33.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:32 smithi099 ceph-mon[37557]: osdmap e93: 8 total, 8 up, 8 in 2024-09-10T18:20:33.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:20:33 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:20:33.870 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_stats PASSED [ 24%] 2024-09-10T18:20:34.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:20:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:20:33] "GET /metrics HTTP/1.1" 200 224651 "" "Prometheus/2.43.0" 2024-09-10T18:20:34.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:33 smithi099 ceph-mon[27037]: pgmap v99: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:34.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:33 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:34.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:33 smithi099 ceph-mon[27037]: osdmap e94: 8 total, 8 up, 8 in 2024-09-10T18:20:34.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:33 smithi099 ceph-mon[37557]: pgmap v99: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:34.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:33 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:34.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:33 smithi099 ceph-mon[37557]: osdmap e94: 8 total, 8 up, 8 in 2024-09-10T18:20:34.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:33 smithi133 ceph-mon[31214]: pgmap v99: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:34.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:33 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:34.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:33 smithi133 ceph-mon[31214]: osdmap e94: 8 total, 8 up, 8 in 2024-09-10T18:20:35.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:34 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:35.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:34 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:35.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:34 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:36.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:35 smithi133 ceph-mon[31214]: pgmap v102: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:36.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:35 smithi133 ceph-mon[31214]: osdmap e95: 8 total, 8 up, 8 in 2024-09-10T18:20:36.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:35 smithi099 ceph-mon[27037]: pgmap v102: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:36.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:35 smithi099 ceph-mon[27037]: osdmap e95: 8 total, 8 up, 8 in 2024-09-10T18:20:36.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:35 smithi099 ceph-mon[37557]: pgmap v102: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:36.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:35 smithi099 ceph-mon[37557]: osdmap e95: 8 total, 8 up, 8 in 2024-09-10T18:20:37.171 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:36 smithi133 ceph-mon[31214]: osdmap e96: 8 total, 8 up, 8 in 2024-09-10T18:20:37.172 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:36 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1943929885' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:37.172 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:36 smithi133 ceph-mon[31214]: from='client.34802 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:37.172 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:36 smithi133 ceph-mon[31214]: pgmap v105: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:37.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:36 smithi099 ceph-mon[27037]: osdmap e96: 8 total, 8 up, 8 in 2024-09-10T18:20:37.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:36 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1943929885' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:37.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:36 smithi099 ceph-mon[27037]: from='client.34802 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:37.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:36 smithi099 ceph-mon[27037]: pgmap v105: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:37.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:36 smithi099 ceph-mon[37557]: osdmap e96: 8 total, 8 up, 8 in 2024-09-10T18:20:37.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:36 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1943929885' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:37.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:36 smithi099 ceph-mon[37557]: from='client.34802 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:37.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:36 smithi099 ceph-mon[37557]: pgmap v105: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:37.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:20:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:20:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:20:37.907 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write PASSED [ 25%] 2024-09-10T18:20:38.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:37 smithi133 ceph-mon[31214]: from='client.34802 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:38.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:37 smithi133 ceph-mon[31214]: osdmap e97: 8 total, 8 up, 8 in 2024-09-10T18:20:38.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:37 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:20:38.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:37 smithi099 ceph-mon[27037]: from='client.34802 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:38.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:37 smithi099 ceph-mon[27037]: osdmap e97: 8 total, 8 up, 8 in 2024-09-10T18:20:38.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:37 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:20:38.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:37 smithi099 ceph-mon[37557]: from='client.34802 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:38.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:37 smithi099 ceph-mon[37557]: osdmap e97: 8 total, 8 up, 8 in 2024-09-10T18:20:38.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:37 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:20:39.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:38 smithi133 ceph-mon[31214]: osdmap e98: 8 total, 8 up, 8 in 2024-09-10T18:20:39.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:38 smithi133 ceph-mon[31214]: pgmap v108: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:39.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:38 smithi099 ceph-mon[27037]: osdmap e98: 8 total, 8 up, 8 in 2024-09-10T18:20:39.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:38 smithi099 ceph-mon[27037]: pgmap v108: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:39.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:38 smithi099 ceph-mon[37557]: osdmap e98: 8 total, 8 up, 8 in 2024-09-10T18:20:39.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:38 smithi099 ceph-mon[37557]: pgmap v108: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:40.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:39 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:40.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:39 smithi133 ceph-mon[31214]: osdmap e99: 8 total, 8 up, 8 in 2024-09-10T18:20:40.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:39 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:40.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:39 smithi099 ceph-mon[27037]: osdmap e99: 8 total, 8 up, 8 in 2024-09-10T18:20:40.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:39 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:40.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:39 smithi099 ceph-mon[37557]: osdmap e99: 8 total, 8 up, 8 in 2024-09-10T18:20:41.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:40 smithi133 ceph-mon[31214]: osdmap e100: 8 total, 8 up, 8 in 2024-09-10T18:20:41.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:40 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/785429142' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:41.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:40 smithi133 ceph-mon[31214]: pgmap v111: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:41.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:40 smithi099 ceph-mon[27037]: osdmap e100: 8 total, 8 up, 8 in 2024-09-10T18:20:41.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:40 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/785429142' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:41.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:40 smithi099 ceph-mon[27037]: pgmap v111: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:41.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:40 smithi099 ceph-mon[37557]: osdmap e100: 8 total, 8 up, 8 in 2024-09-10T18:20:41.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:40 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/785429142' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:41.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:40 smithi099 ceph-mon[37557]: pgmap v111: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:41.939 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_full PASSED [ 26%] 2024-09-10T18:20:42.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:41 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/785429142' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:42.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:41 smithi133 ceph-mon[31214]: osdmap e101: 8 total, 8 up, 8 in 2024-09-10T18:20:42.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:41 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/785429142' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:42.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:41 smithi099 ceph-mon[27037]: osdmap e101: 8 total, 8 up, 8 in 2024-09-10T18:20:42.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:41 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/785429142' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:42.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:41 smithi099 ceph-mon[37557]: osdmap e101: 8 total, 8 up, 8 in 2024-09-10T18:20:43.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:42 smithi133 ceph-mon[31214]: osdmap e102: 8 total, 8 up, 8 in 2024-09-10T18:20:43.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:42 smithi133 ceph-mon[31214]: pgmap v114: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:43.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:42 smithi099 ceph-mon[27037]: osdmap e102: 8 total, 8 up, 8 in 2024-09-10T18:20:43.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:42 smithi099 ceph-mon[27037]: pgmap v114: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:43.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:42 smithi099 ceph-mon[37557]: osdmap e102: 8 total, 8 up, 8 in 2024-09-10T18:20:43.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:42 smithi099 ceph-mon[37557]: pgmap v114: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:43.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:20:43 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:20:43.956 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:20:43 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:20:43] "GET /metrics HTTP/1.1" 200 224805 "" "Prometheus/2.43.0" 2024-09-10T18:20:44.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:43 smithi133 ceph-mon[31214]: osdmap e103: 8 total, 8 up, 8 in 2024-09-10T18:20:44.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:43 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:44.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:43 smithi099 ceph-mon[27037]: osdmap e103: 8 total, 8 up, 8 in 2024-09-10T18:20:44.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:43 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:44.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:43 smithi099 ceph-mon[37557]: osdmap e103: 8 total, 8 up, 8 in 2024-09-10T18:20:44.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:43 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:45.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:44 smithi133 ceph-mon[31214]: osdmap e104: 8 total, 8 up, 8 in 2024-09-10T18:20:45.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:44 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3833819943' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:45.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:44 smithi133 ceph-mon[31214]: from='client.34814 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:45.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:44 smithi133 ceph-mon[31214]: pgmap v117: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:45.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:44 smithi099 ceph-mon[27037]: osdmap e104: 8 total, 8 up, 8 in 2024-09-10T18:20:45.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:44 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3833819943' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:45.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:44 smithi099 ceph-mon[27037]: from='client.34814 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:45.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:44 smithi099 ceph-mon[27037]: pgmap v117: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:45.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:44 smithi099 ceph-mon[37557]: osdmap e104: 8 total, 8 up, 8 in 2024-09-10T18:20:45.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:44 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3833819943' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:45.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:44 smithi099 ceph-mon[37557]: from='client.34814 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:45.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:44 smithi099 ceph-mon[37557]: pgmap v117: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:45.971 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_writesame PASSED [ 28%] 2024-09-10T18:20:46.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:45 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:46.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:45 smithi133 ceph-mon[31214]: from='client.34814 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:46.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:45 smithi133 ceph-mon[31214]: osdmap e105: 8 total, 8 up, 8 in 2024-09-10T18:20:46.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:45 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:46.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:45 smithi099 ceph-mon[27037]: from='client.34814 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:46.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:45 smithi099 ceph-mon[27037]: osdmap e105: 8 total, 8 up, 8 in 2024-09-10T18:20:46.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:45 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:46.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:45 smithi099 ceph-mon[37557]: from='client.34814 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:46.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:45 smithi099 ceph-mon[37557]: osdmap e105: 8 total, 8 up, 8 in 2024-09-10T18:20:47.241 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:20:47 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:20:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:20:47.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:46 smithi133 ceph-mon[31214]: osdmap e106: 8 total, 8 up, 8 in 2024-09-10T18:20:47.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:46 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:47.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:46 smithi133 ceph-mon[31214]: pgmap v120: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:47.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:46 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:47.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:46 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:47.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:46 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:20:47.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:46 smithi099 ceph-mon[27037]: osdmap e106: 8 total, 8 up, 8 in 2024-09-10T18:20:47.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:46 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:47.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:46 smithi099 ceph-mon[27037]: pgmap v120: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:47.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:46 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:47.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:46 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:47.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:46 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:20:47.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:46 smithi099 ceph-mon[37557]: osdmap e106: 8 total, 8 up, 8 in 2024-09-10T18:20:47.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:46 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:20:47.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:46 smithi099 ceph-mon[37557]: pgmap v120: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:47.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:46 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:20:47.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:46 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:20:47.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:46 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:20:48.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:47 smithi133 ceph-mon[31214]: osdmap e107: 8 total, 8 up, 8 in 2024-09-10T18:20:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:47 smithi099 ceph-mon[27037]: osdmap e107: 8 total, 8 up, 8 in 2024-09-10T18:20:48.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:47 smithi099 ceph-mon[37557]: osdmap e107: 8 total, 8 up, 8 in 2024-09-10T18:20:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:48 smithi099 ceph-mon[27037]: osdmap e108: 8 total, 8 up, 8 in 2024-09-10T18:20:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:48 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2045839274' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:48 smithi099 ceph-mon[27037]: pgmap v123: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:20:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:48 smithi099 ceph-mon[37557]: osdmap e108: 8 total, 8 up, 8 in 2024-09-10T18:20:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:48 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2045839274' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:48 smithi099 ceph-mon[37557]: pgmap v123: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:20:49.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:48 smithi133 ceph-mon[31214]: osdmap e108: 8 total, 8 up, 8 in 2024-09-10T18:20:49.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:48 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2045839274' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:49.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:48 smithi133 ceph-mon[31214]: pgmap v123: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:20:50.000 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_append PASSED [ 29%] 2024-09-10T18:20:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:49 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2045839274' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:49 smithi099 ceph-mon[27037]: osdmap e109: 8 total, 8 up, 8 in 2024-09-10T18:20:50.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:50 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2045839274' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:50.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:50 smithi099 ceph-mon[37557]: osdmap e109: 8 total, 8 up, 8 in 2024-09-10T18:20:50.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:49 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2045839274' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:50.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:49 smithi133 ceph-mon[31214]: osdmap e109: 8 total, 8 up, 8 in 2024-09-10T18:20:51.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:51 smithi133 ceph-mon[31214]: osdmap e110: 8 total, 8 up, 8 in 2024-09-10T18:20:51.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:51 smithi133 ceph-mon[31214]: pgmap v126: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:51.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:51 smithi099 ceph-mon[27037]: osdmap e110: 8 total, 8 up, 8 in 2024-09-10T18:20:51.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:51 smithi099 ceph-mon[27037]: pgmap v126: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:51.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:51 smithi099 ceph-mon[37557]: osdmap e110: 8 total, 8 up, 8 in 2024-09-10T18:20:51.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:51 smithi099 ceph-mon[37557]: pgmap v126: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:52.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:52 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:52.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:52 smithi133 ceph-mon[31214]: osdmap e111: 8 total, 8 up, 8 in 2024-09-10T18:20:52.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:52 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:52.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:52 smithi099 ceph-mon[27037]: osdmap e111: 8 total, 8 up, 8 in 2024-09-10T18:20:52.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:52 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:52.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:52 smithi099 ceph-mon[37557]: osdmap e111: 8 total, 8 up, 8 in 2024-09-10T18:20:53.453 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:53 smithi133 ceph-mon[31214]: osdmap e112: 8 total, 8 up, 8 in 2024-09-10T18:20:53.453 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:53 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3443488362' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:53.453 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:53 smithi133 ceph-mon[31214]: pgmap v129: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:53.453 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:53 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:20:53.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:53 smithi099 ceph-mon[27037]: osdmap e112: 8 total, 8 up, 8 in 2024-09-10T18:20:53.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:53 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3443488362' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:53.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:53 smithi099 ceph-mon[27037]: pgmap v129: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:53.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:53 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:20:53.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:53 smithi099 ceph-mon[37557]: osdmap e112: 8 total, 8 up, 8 in 2024-09-10T18:20:53.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:53 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3443488362' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:53.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:53 smithi099 ceph-mon[37557]: pgmap v129: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:20:53.507 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:53 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:20:53.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:20:53 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:20:54.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:20:53 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:20:53] "GET /metrics HTTP/1.1" 200 224832 "" "Prometheus/2.43.0" 2024-09-10T18:20:54.033 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_zeros PASSED [ 30%] 2024-09-10T18:20:54.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:54 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3443488362' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:54.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:54 smithi133 ceph-mon[31214]: osdmap e113: 8 total, 8 up, 8 in 2024-09-10T18:20:54.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:54 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:54.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:54 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3443488362' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:54.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:54 smithi099 ceph-mon[27037]: osdmap e113: 8 total, 8 up, 8 in 2024-09-10T18:20:54.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:54 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:54.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:54 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3443488362' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:54.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:54 smithi099 ceph-mon[37557]: osdmap e113: 8 total, 8 up, 8 in 2024-09-10T18:20:54.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:54 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:20:55.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:55 smithi133 ceph-mon[31214]: osdmap e114: 8 total, 8 up, 8 in 2024-09-10T18:20:55.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:55 smithi133 ceph-mon[31214]: pgmap v132: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:55.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:55 smithi099 ceph-mon[27037]: osdmap e114: 8 total, 8 up, 8 in 2024-09-10T18:20:55.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:55 smithi099 ceph-mon[27037]: pgmap v132: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:55.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:55 smithi099 ceph-mon[37557]: osdmap e114: 8 total, 8 up, 8 in 2024-09-10T18:20:55.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:55 smithi099 ceph-mon[37557]: pgmap v132: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:56.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:56 smithi133 ceph-mon[31214]: osdmap e115: 8 total, 8 up, 8 in 2024-09-10T18:20:56.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:56 smithi099 ceph-mon[27037]: osdmap e115: 8 total, 8 up, 8 in 2024-09-10T18:20:56.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:56 smithi099 ceph-mon[37557]: osdmap e115: 8 total, 8 up, 8 in 2024-09-10T18:20:57.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:57 smithi133 ceph-mon[31214]: osdmap e116: 8 total, 8 up, 8 in 2024-09-10T18:20:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:57 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2879892233' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:57 smithi133 ceph-mon[31214]: pgmap v135: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:57.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:20:57 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:20:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:20:57.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:57 smithi099 ceph-mon[37557]: osdmap e116: 8 total, 8 up, 8 in 2024-09-10T18:20:57.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:57 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2879892233' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:57.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:57 smithi099 ceph-mon[37557]: pgmap v135: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:57.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:57 smithi099 ceph-mon[27037]: osdmap e116: 8 total, 8 up, 8 in 2024-09-10T18:20:57.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:57 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2879892233' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:20:57.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:57 smithi099 ceph-mon[27037]: pgmap v135: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:58.065 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_trunc PASSED [ 31%] 2024-09-10T18:20:58.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:58 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:58.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:58 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2879892233' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:58.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:58 smithi133 ceph-mon[31214]: osdmap e117: 8 total, 8 up, 8 in 2024-09-10T18:20:58.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:58 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:58.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:58 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2879892233' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:58.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:58 smithi099 ceph-mon[27037]: osdmap e117: 8 total, 8 up, 8 in 2024-09-10T18:20:58.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:58 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:20:58.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:58 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2879892233' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:20:58.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:58 smithi099 ceph-mon[37557]: osdmap e117: 8 total, 8 up, 8 in 2024-09-10T18:20:59.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:59 smithi133 ceph-mon[31214]: osdmap e118: 8 total, 8 up, 8 in 2024-09-10T18:20:59.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:20:59 smithi133 ceph-mon[31214]: pgmap v138: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:59.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:59 smithi099 ceph-mon[27037]: osdmap e118: 8 total, 8 up, 8 in 2024-09-10T18:20:59.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:20:59 smithi099 ceph-mon[27037]: pgmap v138: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:20:59.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:59 smithi099 ceph-mon[37557]: osdmap e118: 8 total, 8 up, 8 in 2024-09-10T18:20:59.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:20:59 smithi099 ceph-mon[37557]: pgmap v138: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:00.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:00 smithi133 ceph-mon[31214]: osdmap e119: 8 total, 8 up, 8 in 2024-09-10T18:21:00.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:00 smithi099 ceph-mon[27037]: osdmap e119: 8 total, 8 up, 8 in 2024-09-10T18:21:00.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:00 smithi099 ceph-mon[37557]: osdmap e119: 8 total, 8 up, 8 in 2024-09-10T18:21:01.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:01 smithi133 ceph-mon[31214]: osdmap e120: 8 total, 8 up, 8 in 2024-09-10T18:21:01.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:01 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/19512892' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:01.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:01 smithi133 ceph-mon[31214]: from='client.34838 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:01.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:01 smithi133 ceph-mon[31214]: pgmap v141: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:01.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:01 smithi099 ceph-mon[27037]: osdmap e120: 8 total, 8 up, 8 in 2024-09-10T18:21:01.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:01 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/19512892' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:01.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:01 smithi099 ceph-mon[27037]: from='client.34838 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:01.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:01 smithi099 ceph-mon[27037]: pgmap v141: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:01.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:01 smithi099 ceph-mon[37557]: osdmap e120: 8 total, 8 up, 8 in 2024-09-10T18:21:01.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:01 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/19512892' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:01.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:01 smithi099 ceph-mon[37557]: from='client.34838 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:01.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:01 smithi099 ceph-mon[37557]: pgmap v141: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:02.092 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_cmpext PASSED [ 32%] 2024-09-10T18:21:02.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:02 smithi133 ceph-mon[31214]: from='client.34838 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:02.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:02 smithi133 ceph-mon[31214]: osdmap e121: 8 total, 8 up, 8 in 2024-09-10T18:21:02.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:02 smithi099 ceph-mon[27037]: from='client.34838 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:02.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:02 smithi099 ceph-mon[27037]: osdmap e121: 8 total, 8 up, 8 in 2024-09-10T18:21:02.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:02 smithi099 ceph-mon[37557]: from='client.34838 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:02.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:02 smithi099 ceph-mon[37557]: osdmap e121: 8 total, 8 up, 8 in 2024-09-10T18:21:03.464 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:03 smithi133 ceph-mon[31214]: osdmap e122: 8 total, 8 up, 8 in 2024-09-10T18:21:03.464 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:03 smithi133 ceph-mon[31214]: pgmap v144: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:03.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:03 smithi099 ceph-mon[27037]: osdmap e122: 8 total, 8 up, 8 in 2024-09-10T18:21:03.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:03 smithi099 ceph-mon[27037]: pgmap v144: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:03.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:03 smithi099 ceph-mon[37557]: osdmap e122: 8 total, 8 up, 8 in 2024-09-10T18:21:03.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:03 smithi099 ceph-mon[37557]: pgmap v144: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:03.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:21:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:21:04.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:21:03 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:21:03] "GET /metrics HTTP/1.1" 200 224832 "" "Prometheus/2.43.0" 2024-09-10T18:21:04.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:04 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:04.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:04 smithi133 ceph-mon[31214]: osdmap e123: 8 total, 8 up, 8 in 2024-09-10T18:21:04.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:04 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:04.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:04 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:04.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:04 smithi099 ceph-mon[27037]: osdmap e123: 8 total, 8 up, 8 in 2024-09-10T18:21:04.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:04 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:04.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:04 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:04.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:04 smithi099 ceph-mon[37557]: osdmap e123: 8 total, 8 up, 8 in 2024-09-10T18:21:04.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:04 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:05.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:05 smithi133 ceph-mon[31214]: osdmap e124: 8 total, 8 up, 8 in 2024-09-10T18:21:05.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:05 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3318147114' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:05.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:05 smithi133 ceph-mon[31214]: from='client.34844 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:05.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:05 smithi133 ceph-mon[31214]: pgmap v147: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:05.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:05 smithi099 ceph-mon[27037]: osdmap e124: 8 total, 8 up, 8 in 2024-09-10T18:21:05.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:05 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3318147114' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:05.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:05 smithi099 ceph-mon[27037]: from='client.34844 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:05.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:05 smithi099 ceph-mon[27037]: pgmap v147: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:05.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:05 smithi099 ceph-mon[37557]: osdmap e124: 8 total, 8 up, 8 in 2024-09-10T18:21:05.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:05 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3318147114' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:05.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:05 smithi099 ceph-mon[37557]: from='client.34844 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:05.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:05 smithi099 ceph-mon[37557]: pgmap v147: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:06.123 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_objects_empty PASSED [ 33%] 2024-09-10T18:21:06.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:06 smithi133 ceph-mon[31214]: from='client.34844 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:06.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:06 smithi133 ceph-mon[31214]: osdmap e125: 8 total, 8 up, 8 in 2024-09-10T18:21:06.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:06 smithi099 ceph-mon[27037]: from='client.34844 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:06.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:06 smithi099 ceph-mon[27037]: osdmap e125: 8 total, 8 up, 8 in 2024-09-10T18:21:06.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:06 smithi099 ceph-mon[37557]: from='client.34844 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:06.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:06 smithi099 ceph-mon[37557]: osdmap e125: 8 total, 8 up, 8 in 2024-09-10T18:21:07.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:07 smithi133 ceph-mon[31214]: osdmap e126: 8 total, 8 up, 8 in 2024-09-10T18:21:07.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:07 smithi133 ceph-mon[31214]: pgmap v150: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:07.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:21:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:21:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:21:07.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:07 smithi099 ceph-mon[27037]: osdmap e126: 8 total, 8 up, 8 in 2024-09-10T18:21:07.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:07 smithi099 ceph-mon[27037]: pgmap v150: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:07.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:07 smithi099 ceph-mon[37557]: osdmap e126: 8 total, 8 up, 8 in 2024-09-10T18:21:07.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:07 smithi099 ceph-mon[37557]: pgmap v150: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:08.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:08 smithi133 ceph-mon[31214]: osdmap e127: 8 total, 8 up, 8 in 2024-09-10T18:21:08.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:08 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:21:08.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:08 smithi099 ceph-mon[27037]: osdmap e127: 8 total, 8 up, 8 in 2024-09-10T18:21:08.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:08 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:21:08.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:08 smithi099 ceph-mon[37557]: osdmap e127: 8 total, 8 up, 8 in 2024-09-10T18:21:08.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:08 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:21:09.409 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:09 smithi099 ceph-mon[27037]: osdmap e128: 8 total, 8 up, 8 in 2024-09-10T18:21:09.409 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:09 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1696061536' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:09.409 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:09 smithi099 ceph-mon[27037]: pgmap v153: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:21:09.409 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:09 smithi099 ceph-mon[37557]: osdmap e128: 8 total, 8 up, 8 in 2024-09-10T18:21:09.409 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:09 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1696061536' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:09.410 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:09 smithi099 ceph-mon[37557]: pgmap v153: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:21:09.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:09 smithi133 ceph-mon[31214]: osdmap e128: 8 total, 8 up, 8 in 2024-09-10T18:21:09.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:09 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1696061536' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:09.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:09 smithi133 ceph-mon[31214]: pgmap v153: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:21:10.156 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_objects PASSED [ 34%] 2024-09-10T18:21:10.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:10 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:10.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:10 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1696061536' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:10.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:10 smithi133 ceph-mon[31214]: osdmap e129: 8 total, 8 up, 8 in 2024-09-10T18:21:10.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:10 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:10.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:10 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1696061536' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:10.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:10 smithi099 ceph-mon[27037]: osdmap e129: 8 total, 8 up, 8 in 2024-09-10T18:21:10.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:10 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:10.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:10 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1696061536' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:10.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:10 smithi099 ceph-mon[37557]: osdmap e129: 8 total, 8 up, 8 in 2024-09-10T18:21:11.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:11 smithi133 ceph-mon[31214]: osdmap e130: 8 total, 8 up, 8 in 2024-09-10T18:21:11.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:11 smithi133 ceph-mon[31214]: pgmap v156: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:11.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:11 smithi099 ceph-mon[27037]: osdmap e130: 8 total, 8 up, 8 in 2024-09-10T18:21:11.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:11 smithi099 ceph-mon[27037]: pgmap v156: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:11.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:11 smithi099 ceph-mon[37557]: osdmap e130: 8 total, 8 up, 8 in 2024-09-10T18:21:11.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:11 smithi099 ceph-mon[37557]: pgmap v156: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:12.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:12 smithi133 ceph-mon[31214]: osdmap e131: 8 total, 8 up, 8 in 2024-09-10T18:21:12.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:12 smithi099 ceph-mon[27037]: osdmap e131: 8 total, 8 up, 8 in 2024-09-10T18:21:12.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:12 smithi099 ceph-mon[37557]: osdmap e131: 8 total, 8 up, 8 in 2024-09-10T18:21:13.475 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:13 smithi133 ceph-mon[31214]: osdmap e132: 8 total, 8 up, 8 in 2024-09-10T18:21:13.475 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:13 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2637224935' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:13.475 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:13 smithi133 ceph-mon[31214]: pgmap v159: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:13.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:13 smithi099 ceph-mon[37557]: osdmap e132: 8 total, 8 up, 8 in 2024-09-10T18:21:13.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:13 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2637224935' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:13.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:13 smithi099 ceph-mon[37557]: pgmap v159: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:13.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:13 smithi099 ceph-mon[27037]: osdmap e132: 8 total, 8 up, 8 in 2024-09-10T18:21:13.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:13 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2637224935' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:13.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:13 smithi099 ceph-mon[27037]: pgmap v159: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:13.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:21:13 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:21:14.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:21:13 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:21:13] "GET /metrics HTTP/1.1" 200 224881 "" "Prometheus/2.43.0" 2024-09-10T18:21:14.198 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_ns_objects PASSED [ 35%] 2024-09-10T18:21:14.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:14 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2637224935' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:14.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:14 smithi133 ceph-mon[31214]: osdmap e133: 8 total, 8 up, 8 in 2024-09-10T18:21:14.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:14 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:14.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:14 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2637224935' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:14.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:14 smithi099 ceph-mon[37557]: osdmap e133: 8 total, 8 up, 8 in 2024-09-10T18:21:14.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:14 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:14.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:14 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2637224935' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:14.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:14 smithi099 ceph-mon[27037]: osdmap e133: 8 total, 8 up, 8 in 2024-09-10T18:21:14.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:14 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:15.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:15 smithi133 ceph-mon[31214]: osdmap e134: 8 total, 8 up, 8 in 2024-09-10T18:21:15.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:15 smithi133 ceph-mon[31214]: pgmap v162: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:15.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:15 smithi099 ceph-mon[27037]: osdmap e134: 8 total, 8 up, 8 in 2024-09-10T18:21:15.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:15 smithi099 ceph-mon[27037]: pgmap v162: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:15.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:15 smithi099 ceph-mon[37557]: osdmap e134: 8 total, 8 up, 8 in 2024-09-10T18:21:15.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:15 smithi099 ceph-mon[37557]: pgmap v162: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:16.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:16 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:16.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:16 smithi133 ceph-mon[31214]: osdmap e135: 8 total, 8 up, 8 in 2024-09-10T18:21:16.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:16 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:16.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:16 smithi099 ceph-mon[27037]: osdmap e135: 8 total, 8 up, 8 in 2024-09-10T18:21:16.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:16 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:16.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:16 smithi099 ceph-mon[37557]: osdmap e135: 8 total, 8 up, 8 in 2024-09-10T18:21:17.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:21:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:21:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:21:17.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:17 smithi133 ceph-mon[31214]: osdmap e136: 8 total, 8 up, 8 in 2024-09-10T18:21:17.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:17 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3823061415' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:17.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:17 smithi133 ceph-mon[31214]: from='client.34862 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:17.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:17 smithi133 ceph-mon[31214]: pgmap v165: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:17.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:17 smithi133 ceph-mon[31214]: from='client.34862 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:17.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:17 smithi133 ceph-mon[31214]: osdmap e137: 8 total, 8 up, 8 in 2024-09-10T18:21:17.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:17 smithi099 ceph-mon[27037]: osdmap e136: 8 total, 8 up, 8 in 2024-09-10T18:21:17.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:17 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3823061415' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:17.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:17 smithi099 ceph-mon[27037]: from='client.34862 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:17.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:17 smithi099 ceph-mon[27037]: pgmap v165: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:17.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:17 smithi099 ceph-mon[27037]: from='client.34862 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:17.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:17 smithi099 ceph-mon[27037]: osdmap e137: 8 total, 8 up, 8 in 2024-09-10T18:21:17.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:17 smithi099 ceph-mon[37557]: osdmap e136: 8 total, 8 up, 8 in 2024-09-10T18:21:17.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:17 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3823061415' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:17.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:17 smithi099 ceph-mon[37557]: from='client.34862 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:17.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:17 smithi099 ceph-mon[37557]: pgmap v165: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:17.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:17 smithi099 ceph-mon[37557]: from='client.34862 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:17.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:17 smithi099 ceph-mon[37557]: osdmap e137: 8 total, 8 up, 8 in 2024-09-10T18:21:18.215 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_xattrs PASSED [ 37%] 2024-09-10T18:21:19.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:19 smithi133 ceph-mon[31214]: osdmap e138: 8 total, 8 up, 8 in 2024-09-10T18:21:19.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:19 smithi133 ceph-mon[31214]: pgmap v168: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:19.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:19 smithi099 ceph-mon[27037]: osdmap e138: 8 total, 8 up, 8 in 2024-09-10T18:21:19.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:19 smithi099 ceph-mon[27037]: pgmap v168: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:19.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:19 smithi099 ceph-mon[37557]: osdmap e138: 8 total, 8 up, 8 in 2024-09-10T18:21:19.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:19 smithi099 ceph-mon[37557]: pgmap v168: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:20.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:20 smithi133 ceph-mon[31214]: osdmap e139: 8 total, 8 up, 8 in 2024-09-10T18:21:20.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:20 smithi099 ceph-mon[27037]: osdmap e139: 8 total, 8 up, 8 in 2024-09-10T18:21:20.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:20 smithi099 ceph-mon[37557]: osdmap e139: 8 total, 8 up, 8 in 2024-09-10T18:21:21.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:21 smithi133 ceph-mon[31214]: osdmap e140: 8 total, 8 up, 8 in 2024-09-10T18:21:21.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:21 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1792893302' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:21.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:21 smithi133 ceph-mon[31214]: from='client.34868 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:21.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:21 smithi133 ceph-mon[31214]: pgmap v171: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:21.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:21 smithi099 ceph-mon[27037]: osdmap e140: 8 total, 8 up, 8 in 2024-09-10T18:21:21.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:21 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1792893302' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:21.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:21 smithi099 ceph-mon[27037]: from='client.34868 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:21.508 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:21 smithi099 ceph-mon[27037]: pgmap v171: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:21.508 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:21 smithi099 ceph-mon[37557]: osdmap e140: 8 total, 8 up, 8 in 2024-09-10T18:21:21.508 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:21 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1792893302' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:21.508 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:21 smithi099 ceph-mon[37557]: from='client.34868 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:21.508 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:21 smithi099 ceph-mon[37557]: pgmap v171: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:22.233 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_obj_xattrs PASSED [ 38%] 2024-09-10T18:21:22.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:22 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:22.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:22 smithi133 ceph-mon[31214]: from='client.34868 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:22.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:22 smithi133 ceph-mon[31214]: osdmap e141: 8 total, 8 up, 8 in 2024-09-10T18:21:22.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:22 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:22.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:22 smithi099 ceph-mon[37557]: from='client.34868 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:22.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:22 smithi099 ceph-mon[37557]: osdmap e141: 8 total, 8 up, 8 in 2024-09-10T18:21:22.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:22 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:22.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:22 smithi099 ceph-mon[27037]: from='client.34868 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:22.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:22 smithi099 ceph-mon[27037]: osdmap e141: 8 total, 8 up, 8 in 2024-09-10T18:21:23.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:21:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:21:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:23 smithi133 ceph-mon[31214]: osdmap e142: 8 total, 8 up, 8 in 2024-09-10T18:21:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:23 smithi133 ceph-mon[31214]: pgmap v174: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:21:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:23 smithi099 ceph-mon[27037]: osdmap e142: 8 total, 8 up, 8 in 2024-09-10T18:21:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:23 smithi099 ceph-mon[27037]: pgmap v174: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:21:23.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:23 smithi099 ceph-mon[37557]: osdmap e142: 8 total, 8 up, 8 in 2024-09-10T18:21:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:23 smithi099 ceph-mon[37557]: pgmap v174: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:21:24.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:21:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:21:23] "GET /metrics HTTP/1.1" 200 222560 "" "Prometheus/2.43.0" 2024-09-10T18:21:24.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:24 smithi133 ceph-mon[31214]: osdmap e143: 8 total, 8 up, 8 in 2024-09-10T18:21:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:24 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/632681689' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:24 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:24.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:24 smithi099 ceph-mon[27037]: osdmap e143: 8 total, 8 up, 8 in 2024-09-10T18:21:24.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:24 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/632681689' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:24.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:24 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:24.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:24 smithi099 ceph-mon[37557]: osdmap e143: 8 total, 8 up, 8 in 2024-09-10T18:21:24.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:24 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/632681689' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:24.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:24 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:25.256 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_pool_id PASSED [ 39%] 2024-09-10T18:21:25.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:25 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/632681689' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:25.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:25 smithi099 ceph-mon[27037]: osdmap e144: 8 total, 8 up, 8 in 2024-09-10T18:21:25.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:25 smithi099 ceph-mon[27037]: pgmap v177: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:25.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:25 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/632681689' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:25.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:25 smithi099 ceph-mon[37557]: osdmap e144: 8 total, 8 up, 8 in 2024-09-10T18:21:25.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:25 smithi099 ceph-mon[37557]: pgmap v177: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:25.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:25 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/632681689' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:25.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:25 smithi133 ceph-mon[31214]: osdmap e144: 8 total, 8 up, 8 in 2024-09-10T18:21:25.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:25 smithi133 ceph-mon[31214]: pgmap v177: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:26.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:26 smithi099 ceph-mon[27037]: osdmap e145: 8 total, 8 up, 8 in 2024-09-10T18:21:26.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:26 smithi099 ceph-mon[37557]: osdmap e145: 8 total, 8 up, 8 in 2024-09-10T18:21:26.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:26 smithi133 ceph-mon[31214]: osdmap e145: 8 total, 8 up, 8 in 2024-09-10T18:21:27.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:21:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:21:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:21:27.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:27 smithi133 ceph-mon[31214]: osdmap e146: 8 total, 8 up, 8 in 2024-09-10T18:21:27.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:27 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3518716491' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:27.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:27 smithi133 ceph-mon[31214]: from='client.34880 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:27.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:27 smithi133 ceph-mon[31214]: pgmap v180: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:27.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:27 smithi099 ceph-mon[27037]: osdmap e146: 8 total, 8 up, 8 in 2024-09-10T18:21:27.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:27 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3518716491' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:27.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:27 smithi099 ceph-mon[27037]: from='client.34880 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:27.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:27 smithi099 ceph-mon[27037]: pgmap v180: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:27.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:27 smithi099 ceph-mon[37557]: osdmap e146: 8 total, 8 up, 8 in 2024-09-10T18:21:27.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:27 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3518716491' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:27.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:27 smithi099 ceph-mon[37557]: from='client.34880 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:27.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:27 smithi099 ceph-mon[37557]: pgmap v180: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:28.284 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_pool_name PASSED [ 40%] 2024-09-10T18:21:28.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:28 smithi133 ceph-mon[31214]: from='client.34880 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:28.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:28 smithi133 ceph-mon[31214]: osdmap e147: 8 total, 8 up, 8 in 2024-09-10T18:21:28.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:28 smithi099 ceph-mon[27037]: from='client.34880 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:28.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:28 smithi099 ceph-mon[27037]: osdmap e147: 8 total, 8 up, 8 in 2024-09-10T18:21:28.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:28 smithi099 ceph-mon[37557]: from='client.34880 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:28.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:28 smithi099 ceph-mon[37557]: osdmap e147: 8 total, 8 up, 8 in 2024-09-10T18:21:29.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:29 smithi133 ceph-mon[31214]: osdmap e148: 8 total, 8 up, 8 in 2024-09-10T18:21:29.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:29 smithi133 ceph-mon[31214]: pgmap v183: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:29.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:29 smithi099 ceph-mon[27037]: osdmap e148: 8 total, 8 up, 8 in 2024-09-10T18:21:29.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:29 smithi099 ceph-mon[27037]: pgmap v183: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:29.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:29 smithi099 ceph-mon[37557]: osdmap e148: 8 total, 8 up, 8 in 2024-09-10T18:21:29.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:29 smithi099 ceph-mon[37557]: pgmap v183: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:30.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:30 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:30.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:30 smithi133 ceph-mon[31214]: osdmap e149: 8 total, 8 up, 8 in 2024-09-10T18:21:30.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:30 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:30.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:30 smithi099 ceph-mon[27037]: osdmap e149: 8 total, 8 up, 8 in 2024-09-10T18:21:30.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:30 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:30.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:30 smithi099 ceph-mon[37557]: osdmap e149: 8 total, 8 up, 8 in 2024-09-10T18:21:31.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:31 smithi133 ceph-mon[31214]: osdmap e150: 8 total, 8 up, 8 in 2024-09-10T18:21:31.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:31 smithi133 ceph-mon[31214]: pgmap v186: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:31 smithi099 ceph-mon[27037]: osdmap e150: 8 total, 8 up, 8 in 2024-09-10T18:21:31.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:31 smithi099 ceph-mon[27037]: pgmap v186: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:31 smithi099 ceph-mon[37557]: osdmap e150: 8 total, 8 up, 8 in 2024-09-10T18:21:31.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:31 smithi099 ceph-mon[37557]: pgmap v186: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:32.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:32 smithi133 ceph-mon[31214]: osdmap e151: 8 total, 8 up, 8 in 2024-09-10T18:21:32.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:32 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/816134410' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:32.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:32 smithi133 ceph-mon[31214]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:32.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:32 smithi099 ceph-mon[27037]: osdmap e151: 8 total, 8 up, 8 in 2024-09-10T18:21:32.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:32 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/816134410' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:32.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:32 smithi099 ceph-mon[27037]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:32.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:32 smithi099 ceph-mon[37557]: osdmap e151: 8 total, 8 up, 8 in 2024-09-10T18:21:32.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:32 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/816134410' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:32.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:32 smithi099 ceph-mon[37557]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:33.302 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_create_snap PASSED [ 41%] 2024-09-10T18:21:33.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:33 smithi099 ceph-mon[27037]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:33.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:33 smithi099 ceph-mon[27037]: osdmap e152: 8 total, 8 up, 8 in 2024-09-10T18:21:33.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:33 smithi099 ceph-mon[27037]: pgmap v189: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:33.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:33 smithi099 ceph-mon[37557]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:33.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:33 smithi099 ceph-mon[37557]: osdmap e152: 8 total, 8 up, 8 in 2024-09-10T18:21:33.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:33 smithi099 ceph-mon[37557]: pgmap v189: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:33.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:33 smithi133 ceph-mon[31214]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:33.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:33 smithi133 ceph-mon[31214]: osdmap e152: 8 total, 8 up, 8 in 2024-09-10T18:21:33.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:33 smithi133 ceph-mon[31214]: pgmap v189: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:33.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:21:33 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:21:34.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:21:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:21:33] "GET /metrics HTTP/1.1" 200 222560 "" "Prometheus/2.43.0" 2024-09-10T18:21:34.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:34 smithi133 ceph-mon[31214]: osdmap e153: 8 total, 8 up, 8 in 2024-09-10T18:21:34.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:34 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:34.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:34 smithi099 ceph-mon[27037]: osdmap e153: 8 total, 8 up, 8 in 2024-09-10T18:21:34.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:34 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:34.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:34 smithi099 ceph-mon[37557]: osdmap e153: 8 total, 8 up, 8 in 2024-09-10T18:21:34.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:34 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:35.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:35 smithi133 ceph-mon[31214]: osdmap e154: 8 total, 8 up, 8 in 2024-09-10T18:21:35.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:35 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2957779499' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:35.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:35 smithi133 ceph-mon[31214]: pgmap v192: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:35.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:35 smithi099 ceph-mon[27037]: osdmap e154: 8 total, 8 up, 8 in 2024-09-10T18:21:35.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:35 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2957779499' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:35.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:35 smithi099 ceph-mon[27037]: pgmap v192: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:35.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:35 smithi099 ceph-mon[37557]: osdmap e154: 8 total, 8 up, 8 in 2024-09-10T18:21:35.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:35 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2957779499' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:35.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:35 smithi099 ceph-mon[37557]: pgmap v192: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:36.340 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_snaps_empty PASSED [ 42%] 2024-09-10T18:21:36.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:36 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2957779499' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:36.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:36 smithi133 ceph-mon[31214]: osdmap e155: 8 total, 8 up, 8 in 2024-09-10T18:21:36.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:36 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2957779499' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:36.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:36 smithi099 ceph-mon[27037]: osdmap e155: 8 total, 8 up, 8 in 2024-09-10T18:21:36.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:36 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2957779499' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:36.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:36 smithi099 ceph-mon[37557]: osdmap e155: 8 total, 8 up, 8 in 2024-09-10T18:21:37.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:37 smithi133 ceph-mon[31214]: osdmap e156: 8 total, 8 up, 8 in 2024-09-10T18:21:37.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:37 smithi133 ceph-mon[31214]: pgmap v195: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:37.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:21:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:21:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:21:37.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:37 smithi099 ceph-mon[27037]: osdmap e156: 8 total, 8 up, 8 in 2024-09-10T18:21:37.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:37 smithi099 ceph-mon[27037]: pgmap v195: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:37.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:37 smithi099 ceph-mon[37557]: osdmap e156: 8 total, 8 up, 8 in 2024-09-10T18:21:37.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:37 smithi099 ceph-mon[37557]: pgmap v195: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:38.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:38 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:38.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:38 smithi133 ceph-mon[31214]: osdmap e157: 8 total, 8 up, 8 in 2024-09-10T18:21:38.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:38 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:21:38.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:38 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:38.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:38 smithi099 ceph-mon[27037]: osdmap e157: 8 total, 8 up, 8 in 2024-09-10T18:21:38.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:38 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:21:38.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:38 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:38.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:38 smithi099 ceph-mon[37557]: osdmap e157: 8 total, 8 up, 8 in 2024-09-10T18:21:38.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:38 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:21:39.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:39 smithi099 ceph-mon[27037]: osdmap e158: 8 total, 8 up, 8 in 2024-09-10T18:21:39.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:39 smithi099 ceph-mon[27037]: pgmap v198: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:39.657 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:39 smithi099 ceph-mon[37557]: osdmap e158: 8 total, 8 up, 8 in 2024-09-10T18:21:39.657 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:39 smithi099 ceph-mon[37557]: pgmap v198: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:39.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:39 smithi133 ceph-mon[31214]: osdmap e158: 8 total, 8 up, 8 in 2024-09-10T18:21:39.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:39 smithi133 ceph-mon[31214]: pgmap v198: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:40.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:40 smithi133 ceph-mon[31214]: osdmap e159: 8 total, 8 up, 8 in 2024-09-10T18:21:40.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:40 smithi099 ceph-mon[27037]: osdmap e159: 8 total, 8 up, 8 in 2024-09-10T18:21:40.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:40 smithi099 ceph-mon[37557]: osdmap e159: 8 total, 8 up, 8 in 2024-09-10T18:21:41.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:41 smithi133 ceph-mon[31214]: pgmap v201: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:41.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:41 smithi133 ceph-mon[31214]: osdmap e160: 8 total, 8 up, 8 in 2024-09-10T18:21:41.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:41 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1538963117' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:41.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:41 smithi133 ceph-mon[31214]: from='client.34898 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:41.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:41 smithi099 ceph-mon[27037]: pgmap v201: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:41.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:41 smithi099 ceph-mon[27037]: osdmap e160: 8 total, 8 up, 8 in 2024-09-10T18:21:41.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:41 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1538963117' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:41.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:41 smithi099 ceph-mon[27037]: from='client.34898 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:41.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:41 smithi099 ceph-mon[37557]: pgmap v201: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:41.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:41 smithi099 ceph-mon[37557]: osdmap e160: 8 total, 8 up, 8 in 2024-09-10T18:21:41.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:41 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1538963117' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:41.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:41 smithi099 ceph-mon[37557]: from='client.34898 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:42.387 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_snaps PASSED [ 43%] 2024-09-10T18:21:42.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:42 smithi133 ceph-mon[31214]: from='client.34898 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:42.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:42 smithi133 ceph-mon[31214]: osdmap e161: 8 total, 8 up, 8 in 2024-09-10T18:21:42.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:42 smithi099 ceph-mon[27037]: from='client.34898 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:42.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:42 smithi099 ceph-mon[27037]: osdmap e161: 8 total, 8 up, 8 in 2024-09-10T18:21:42.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:42 smithi099 ceph-mon[37557]: from='client.34898 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:42.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:42 smithi099 ceph-mon[37557]: osdmap e161: 8 total, 8 up, 8 in 2024-09-10T18:21:43.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:43 smithi133 ceph-mon[31214]: pgmap v203: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:43.642 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:43 smithi133 ceph-mon[31214]: osdmap e162: 8 total, 8 up, 8 in 2024-09-10T18:21:43.642 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:21:43 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:21:43.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:43 smithi099 ceph-mon[27037]: pgmap v203: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:43.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:43 smithi099 ceph-mon[27037]: osdmap e162: 8 total, 8 up, 8 in 2024-09-10T18:21:43.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:43 smithi099 ceph-mon[37557]: pgmap v203: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:43.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:43 smithi099 ceph-mon[37557]: osdmap e162: 8 total, 8 up, 8 in 2024-09-10T18:21:44.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:21:43 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:21:43] "GET /metrics HTTP/1.1" 200 224909 "" "Prometheus/2.43.0" 2024-09-10T18:21:44.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:44 smithi133 ceph-mon[31214]: osdmap e163: 8 total, 8 up, 8 in 2024-09-10T18:21:44.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:44 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:44.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:44 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:44.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:44 smithi099 ceph-mon[27037]: osdmap e163: 8 total, 8 up, 8 in 2024-09-10T18:21:44.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:44 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:44.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:44 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:44.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:44 smithi099 ceph-mon[37557]: osdmap e163: 8 total, 8 up, 8 in 2024-09-10T18:21:44.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:44 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:44.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:44 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:45.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:45 smithi133 ceph-mon[31214]: pgmap v206: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:45.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:45 smithi133 ceph-mon[31214]: osdmap e164: 8 total, 8 up, 8 in 2024-09-10T18:21:45.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:45 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/484211879' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:45.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:45 smithi133 ceph-mon[31214]: from='client.34904 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:45.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:45 smithi099 ceph-mon[27037]: pgmap v206: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:45.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:45 smithi099 ceph-mon[27037]: osdmap e164: 8 total, 8 up, 8 in 2024-09-10T18:21:45.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:45 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/484211879' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:45.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:45 smithi099 ceph-mon[27037]: from='client.34904 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:45.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:45 smithi099 ceph-mon[37557]: pgmap v206: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:45.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:45 smithi099 ceph-mon[37557]: osdmap e164: 8 total, 8 up, 8 in 2024-09-10T18:21:45.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:45 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/484211879' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:45.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:45 smithi099 ceph-mon[37557]: from='client.34904 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:46.402 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_lookup_snap PASSED [ 44%] 2024-09-10T18:21:46.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:46 smithi133 ceph-mon[31214]: from='client.34904 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:46 smithi133 ceph-mon[31214]: osdmap e165: 8 total, 8 up, 8 in 2024-09-10T18:21:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:46 smithi133 ceph-mon[31214]: osdmap e166: 8 total, 8 up, 8 in 2024-09-10T18:21:46.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:46 smithi099 ceph-mon[27037]: from='client.34904 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:46.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:46 smithi099 ceph-mon[27037]: osdmap e165: 8 total, 8 up, 8 in 2024-09-10T18:21:46.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:46 smithi099 ceph-mon[27037]: osdmap e166: 8 total, 8 up, 8 in 2024-09-10T18:21:46.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:46 smithi099 ceph-mon[37557]: from='client.34904 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:46.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:46 smithi099 ceph-mon[37557]: osdmap e165: 8 total, 8 up, 8 in 2024-09-10T18:21:46.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:46 smithi099 ceph-mon[37557]: osdmap e166: 8 total, 8 up, 8 in 2024-09-10T18:21:47.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:21:47 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:21:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:21:47.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:47 smithi133 ceph-mon[31214]: pgmap v209: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:47.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:47 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:47.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:47 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:47.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:47 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:47.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:47 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:21:47.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:47 smithi099 ceph-mon[27037]: pgmap v209: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:47.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:47 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:47.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:47 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:47.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:47 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:47.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:47 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:21:47.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:47 smithi099 ceph-mon[37557]: pgmap v209: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:47.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:47 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:21:47.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:47 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:21:47.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:47 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:21:47.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:47 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:21:48.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:48 smithi133 ceph-mon[31214]: osdmap e167: 8 total, 8 up, 8 in 2024-09-10T18:21:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:48 smithi099 ceph-mon[27037]: osdmap e167: 8 total, 8 up, 8 in 2024-09-10T18:21:48.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:48 smithi099 ceph-mon[37557]: osdmap e167: 8 total, 8 up, 8 in 2024-09-10T18:21:49.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:49 smithi133 ceph-mon[31214]: pgmap v212: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:49.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:49 smithi133 ceph-mon[31214]: osdmap e168: 8 total, 8 up, 8 in 2024-09-10T18:21:49.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:49 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3432360407' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:49.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:49 smithi133 ceph-mon[31214]: from='client.34910 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:49 smithi099 ceph-mon[27037]: pgmap v212: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:49 smithi099 ceph-mon[27037]: osdmap e168: 8 total, 8 up, 8 in 2024-09-10T18:21:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:49 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3432360407' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:49 smithi099 ceph-mon[27037]: from='client.34910 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:49.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:49 smithi099 ceph-mon[37557]: pgmap v212: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:49.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:49 smithi099 ceph-mon[37557]: osdmap e168: 8 total, 8 up, 8 in 2024-09-10T18:21:49.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:49 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3432360407' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:49.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:49 smithi099 ceph-mon[37557]: from='client.34910 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:50.444 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_timestamp PASSED [ 46%] 2024-09-10T18:21:50.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:50 smithi133 ceph-mon[31214]: from='client.34910 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:50.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:50 smithi133 ceph-mon[31214]: osdmap e169: 8 total, 8 up, 8 in 2024-09-10T18:21:50.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:50 smithi099 ceph-mon[27037]: from='client.34910 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:50.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:50 smithi099 ceph-mon[27037]: osdmap e169: 8 total, 8 up, 8 in 2024-09-10T18:21:50.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:50 smithi099 ceph-mon[37557]: from='client.34910 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:50.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:50 smithi099 ceph-mon[37557]: osdmap e169: 8 total, 8 up, 8 in 2024-09-10T18:21:51.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:51 smithi133 ceph-mon[31214]: pgmap v215: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:51.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:51 smithi133 ceph-mon[31214]: osdmap e170: 8 total, 8 up, 8 in 2024-09-10T18:21:51.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:51 smithi099 ceph-mon[27037]: pgmap v215: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:51.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:51 smithi099 ceph-mon[27037]: osdmap e170: 8 total, 8 up, 8 in 2024-09-10T18:21:51.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:51 smithi099 ceph-mon[37557]: pgmap v215: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:51.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:51 smithi099 ceph-mon[37557]: osdmap e170: 8 total, 8 up, 8 in 2024-09-10T18:21:52.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:52 smithi133 ceph-mon[31214]: osdmap e171: 8 total, 8 up, 8 in 2024-09-10T18:21:52.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:52 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:21:52.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:52 smithi133 ceph-mon[31214]: osdmap e172: 8 total, 8 up, 8 in 2024-09-10T18:21:52.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:52 smithi099 ceph-mon[27037]: osdmap e171: 8 total, 8 up, 8 in 2024-09-10T18:21:52.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:52 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:21:52.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:52 smithi099 ceph-mon[27037]: osdmap e172: 8 total, 8 up, 8 in 2024-09-10T18:21:52.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:52 smithi099 ceph-mon[37557]: osdmap e171: 8 total, 8 up, 8 in 2024-09-10T18:21:52.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:52 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:21:52.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:52 smithi099 ceph-mon[37557]: osdmap e172: 8 total, 8 up, 8 in 2024-09-10T18:21:53.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:21:53 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:21:53.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:53 smithi133 ceph-mon[31214]: pgmap v218: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:53.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:53 smithi133 ceph-mon[31214]: osdmap e173: 8 total, 8 up, 8 in 2024-09-10T18:21:53.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:53 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3142959694' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:53.755 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:21:53 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:21:53] "GET /metrics HTTP/1.1" 200 224983 "" "Prometheus/2.43.0" 2024-09-10T18:21:53.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:53 smithi099 ceph-mon[27037]: pgmap v218: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:53.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:53 smithi099 ceph-mon[27037]: osdmap e173: 8 total, 8 up, 8 in 2024-09-10T18:21:53.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:53 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3142959694' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:53.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:53 smithi099 ceph-mon[37557]: pgmap v218: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:21:53.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:53 smithi099 ceph-mon[37557]: osdmap e173: 8 total, 8 up, 8 in 2024-09-10T18:21:53.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:53 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3142959694' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:21:54.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:54 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:54.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:54 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3142959694' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:54.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:54 smithi133 ceph-mon[31214]: osdmap e174: 8 total, 8 up, 8 in 2024-09-10T18:21:54.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:54 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:54.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:54 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3142959694' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:54.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:54 smithi099 ceph-mon[27037]: osdmap e174: 8 total, 8 up, 8 in 2024-09-10T18:21:54.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:54 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:21:54.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:54 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3142959694' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:21:54.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:54 smithi099 ceph-mon[37557]: osdmap e174: 8 total, 8 up, 8 in 2024-09-10T18:21:55.448 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_remove_snap PASSED [ 47%] 2024-09-10T18:21:55.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:55 smithi133 ceph-mon[31214]: pgmap v221: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:55.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:55 smithi133 ceph-mon[31214]: osdmap e175: 8 total, 8 up, 8 in 2024-09-10T18:21:56.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:55 smithi099 ceph-mon[27037]: pgmap v221: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:56.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:55 smithi099 ceph-mon[27037]: osdmap e175: 8 total, 8 up, 8 in 2024-09-10T18:21:56.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:55 smithi099 ceph-mon[37557]: pgmap v221: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:56.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:55 smithi099 ceph-mon[37557]: osdmap e175: 8 total, 8 up, 8 in 2024-09-10T18:21:57.444 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:21:57 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:21:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:21:57.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:57 smithi133 ceph-mon[31214]: pgmap v224: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:57.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:57 smithi133 ceph-mon[31214]: osdmap e176: 8 total, 8 up, 8 in 2024-09-10T18:21:57.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:57 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:57.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:57 smithi099 ceph-mon[27037]: pgmap v224: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:57.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:57 smithi099 ceph-mon[27037]: osdmap e176: 8 total, 8 up, 8 in 2024-09-10T18:21:57.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:57 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:57.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:57 smithi099 ceph-mon[37557]: pgmap v224: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:21:57.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:57 smithi099 ceph-mon[37557]: osdmap e176: 8 total, 8 up, 8 in 2024-09-10T18:21:57.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:57 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:21:58.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:58 smithi133 ceph-mon[31214]: osdmap e177: 8 total, 8 up, 8 in 2024-09-10T18:21:58.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:58 smithi099 ceph-mon[27037]: osdmap e177: 8 total, 8 up, 8 in 2024-09-10T18:21:58.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:58 smithi099 ceph-mon[37557]: osdmap e177: 8 total, 8 up, 8 in 2024-09-10T18:21:59.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:59 smithi133 ceph-mon[31214]: pgmap v227: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:21:59.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:21:59 smithi133 ceph-mon[31214]: osdmap e178: 8 total, 8 up, 8 in 2024-09-10T18:21:59.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:59 smithi099 ceph-mon[27037]: pgmap v227: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:21:59.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:21:59 smithi099 ceph-mon[27037]: osdmap e178: 8 total, 8 up, 8 in 2024-09-10T18:21:59.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:59 smithi099 ceph-mon[37557]: pgmap v227: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:21:59.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:21:59 smithi099 ceph-mon[37557]: osdmap e178: 8 total, 8 up, 8 in 2024-09-10T18:22:00.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:00 smithi133 ceph-mon[31214]: osdmap e179: 8 total, 8 up, 8 in 2024-09-10T18:22:00.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:00 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/418319551' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:00.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:00 smithi133 ceph-mon[31214]: from='client.34922 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:00.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:00 smithi099 ceph-mon[27037]: osdmap e179: 8 total, 8 up, 8 in 2024-09-10T18:22:00.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:00 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/418319551' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:00.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:00 smithi099 ceph-mon[27037]: from='client.34922 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:00.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:00 smithi099 ceph-mon[37557]: osdmap e179: 8 total, 8 up, 8 in 2024-09-10T18:22:00.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:00 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/418319551' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:00.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:00 smithi099 ceph-mon[37557]: from='client.34922 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:01.490 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_rollback PASSED [ 48%] 2024-09-10T18:22:01.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:01 smithi133 ceph-mon[31214]: pgmap v230: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:22:01.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:01 smithi133 ceph-mon[31214]: from='client.34922 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:01.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:01 smithi133 ceph-mon[31214]: osdmap e180: 8 total, 8 up, 8 in 2024-09-10T18:22:01.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:01 smithi099 ceph-mon[27037]: pgmap v230: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:22:01.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:01 smithi099 ceph-mon[27037]: from='client.34922 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:01.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:01 smithi099 ceph-mon[27037]: osdmap e180: 8 total, 8 up, 8 in 2024-09-10T18:22:01.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:01 smithi099 ceph-mon[37557]: pgmap v230: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:22:01.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:01 smithi099 ceph-mon[37557]: from='client.34922 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:01.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:01 smithi099 ceph-mon[37557]: osdmap e180: 8 total, 8 up, 8 in 2024-09-10T18:22:02.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:02 smithi099 ceph-mon[27037]: osdmap e181: 8 total, 8 up, 8 in 2024-09-10T18:22:02.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:02 smithi099 ceph-mon[37557]: osdmap e181: 8 total, 8 up, 8 in 2024-09-10T18:22:02.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:02 smithi133 ceph-mon[31214]: osdmap e181: 8 total, 8 up, 8 in 2024-09-10T18:22:03.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:03 smithi099 ceph-mon[27037]: pgmap v233: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:03.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:03 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:03.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:03 smithi099 ceph-mon[27037]: osdmap e182: 8 total, 8 up, 8 in 2024-09-10T18:22:03.756 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:22:03 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:22:03] "GET /metrics HTTP/1.1" 200 224983 "" "Prometheus/2.43.0" 2024-09-10T18:22:03.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:03 smithi099 ceph-mon[37557]: pgmap v233: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:03.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:03 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:03.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:03 smithi099 ceph-mon[37557]: osdmap e182: 8 total, 8 up, 8 in 2024-09-10T18:22:03.991 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:22:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:22:03.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:03 smithi133 ceph-mon[31214]: pgmap v233: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:03.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:03 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:03.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:03 smithi133 ceph-mon[31214]: osdmap e182: 8 total, 8 up, 8 in 2024-09-10T18:22:04.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:04 smithi133 ceph-mon[31214]: osdmap e183: 8 total, 8 up, 8 in 2024-09-10T18:22:04.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:04 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:05.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:04 smithi099 ceph-mon[27037]: osdmap e183: 8 total, 8 up, 8 in 2024-09-10T18:22:05.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:04 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:05.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:04 smithi099 ceph-mon[37557]: osdmap e183: 8 total, 8 up, 8 in 2024-09-10T18:22:05.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:04 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:05.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:05 smithi133 ceph-mon[31214]: pgmap v236: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:05.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:05 smithi133 ceph-mon[31214]: osdmap e184: 8 total, 8 up, 8 in 2024-09-10T18:22:06.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:05 smithi099 ceph-mon[27037]: pgmap v236: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:06.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:05 smithi099 ceph-mon[27037]: osdmap e184: 8 total, 8 up, 8 in 2024-09-10T18:22:06.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:05 smithi099 ceph-mon[37557]: pgmap v236: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:06.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:05 smithi099 ceph-mon[37557]: osdmap e184: 8 total, 8 up, 8 in 2024-09-10T18:22:06.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:06 smithi133 ceph-mon[31214]: osdmap e185: 8 total, 8 up, 8 in 2024-09-10T18:22:06.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:06 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3310258896' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:06.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:06 smithi133 ceph-mon[31214]: from='client.34928 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:07.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:06 smithi099 ceph-mon[27037]: osdmap e185: 8 total, 8 up, 8 in 2024-09-10T18:22:07.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:06 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3310258896' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:07.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:06 smithi099 ceph-mon[27037]: from='client.34928 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:07.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:06 smithi099 ceph-mon[37557]: osdmap e185: 8 total, 8 up, 8 in 2024-09-10T18:22:07.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:06 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3310258896' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:07.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:06 smithi099 ceph-mon[37557]: from='client.34928 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:07.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:22:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:22:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:22:07.541 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_read PASSED [ 49%] 2024-09-10T18:22:07.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:07 smithi133 ceph-mon[31214]: pgmap v239: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:07.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:07 smithi133 ceph-mon[31214]: from='client.34928 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:07.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:07 smithi133 ceph-mon[31214]: osdmap e186: 8 total, 8 up, 8 in 2024-09-10T18:22:07.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:07 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:08.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:07 smithi099 ceph-mon[27037]: pgmap v239: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:08.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:07 smithi099 ceph-mon[27037]: from='client.34928 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:08.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:07 smithi099 ceph-mon[27037]: osdmap e186: 8 total, 8 up, 8 in 2024-09-10T18:22:08.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:07 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:08.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:07 smithi099 ceph-mon[37557]: pgmap v239: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:08.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:07 smithi099 ceph-mon[37557]: from='client.34928 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:08.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:07 smithi099 ceph-mon[37557]: osdmap e186: 8 total, 8 up, 8 in 2024-09-10T18:22:08.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:07 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:08.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:08 smithi133 ceph-mon[31214]: osdmap e187: 8 total, 8 up, 8 in 2024-09-10T18:22:09.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:08 smithi099 ceph-mon[27037]: osdmap e187: 8 total, 8 up, 8 in 2024-09-10T18:22:09.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:08 smithi099 ceph-mon[37557]: osdmap e187: 8 total, 8 up, 8 in 2024-09-10T18:22:09.907 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:09 smithi099 ceph-mon[27037]: pgmap v242: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:09.907 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:09 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:09.907 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:09 smithi099 ceph-mon[27037]: osdmap e188: 8 total, 8 up, 8 in 2024-09-10T18:22:09.907 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:09 smithi099 ceph-mon[37557]: pgmap v242: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:09.907 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:09 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:09.907 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:09 smithi099 ceph-mon[37557]: osdmap e188: 8 total, 8 up, 8 in 2024-09-10T18:22:09.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:09 smithi133 ceph-mon[31214]: pgmap v242: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:09.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:09 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:09.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:09 smithi133 ceph-mon[31214]: osdmap e188: 8 total, 8 up, 8 in 2024-09-10T18:22:10.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:10 smithi133 ceph-mon[31214]: osdmap e189: 8 total, 8 up, 8 in 2024-09-10T18:22:10.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:10 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1869960797' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:11.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:10 smithi099 ceph-mon[27037]: osdmap e189: 8 total, 8 up, 8 in 2024-09-10T18:22:11.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:10 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1869960797' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:11.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:10 smithi099 ceph-mon[37557]: osdmap e189: 8 total, 8 up, 8 in 2024-09-10T18:22:11.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:10 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1869960797' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:11.565 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_set_omap PASSED [ 50%] 2024-09-10T18:22:11.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:11 smithi133 ceph-mon[31214]: pgmap v245: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:11.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:11 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1869960797' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:11.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:11 smithi133 ceph-mon[31214]: osdmap e190: 8 total, 8 up, 8 in 2024-09-10T18:22:12.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:11 smithi099 ceph-mon[27037]: pgmap v245: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:12.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:11 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1869960797' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:12.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:11 smithi099 ceph-mon[27037]: osdmap e190: 8 total, 8 up, 8 in 2024-09-10T18:22:12.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:11 smithi099 ceph-mon[37557]: pgmap v245: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:12.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:11 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1869960797' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:12.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:11 smithi099 ceph-mon[37557]: osdmap e190: 8 total, 8 up, 8 in 2024-09-10T18:22:12.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:12 smithi133 ceph-mon[31214]: osdmap e191: 8 total, 8 up, 8 in 2024-09-10T18:22:13.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:12 smithi099 ceph-mon[27037]: osdmap e191: 8 total, 8 up, 8 in 2024-09-10T18:22:13.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:12 smithi099 ceph-mon[37557]: osdmap e191: 8 total, 8 up, 8 in 2024-09-10T18:22:13.892 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:13 smithi133 ceph-mon[31214]: pgmap v248: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:13.892 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:13 smithi133 ceph-mon[31214]: osdmap e192: 8 total, 8 up, 8 in 2024-09-10T18:22:13.893 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:22:13 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:22:14.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:13 smithi099 ceph-mon[27037]: pgmap v248: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:14.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:13 smithi099 ceph-mon[27037]: osdmap e192: 8 total, 8 up, 8 in 2024-09-10T18:22:14.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:22:13 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:22:13] "GET /metrics HTTP/1.1" 200 224983 "" "Prometheus/2.43.0" 2024-09-10T18:22:14.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:13 smithi099 ceph-mon[37557]: pgmap v248: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:14.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:13 smithi099 ceph-mon[37557]: osdmap e192: 8 total, 8 up, 8 in 2024-09-10T18:22:14.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:14 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:14.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:14 smithi133 ceph-mon[31214]: osdmap e193: 8 total, 8 up, 8 in 2024-09-10T18:22:14.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:14 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1567830385' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:14.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:14 smithi133 ceph-mon[31214]: from='client.34940 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:15.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:14 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:15.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:14 smithi099 ceph-mon[27037]: osdmap e193: 8 total, 8 up, 8 in 2024-09-10T18:22:15.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:14 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1567830385' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:15.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:14 smithi099 ceph-mon[27037]: from='client.34940 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:15.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:14 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:15.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:14 smithi099 ceph-mon[37557]: osdmap e193: 8 total, 8 up, 8 in 2024-09-10T18:22:15.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:14 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1567830385' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:15.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:14 smithi099 ceph-mon[37557]: from='client.34940 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:15.603 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_set_omap_aio PASSED [ 51%] 2024-09-10T18:22:15.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:15 smithi133 ceph-mon[31214]: pgmap v251: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:15.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:15 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:15.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:15 smithi133 ceph-mon[31214]: from='client.34940 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:15.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:15 smithi133 ceph-mon[31214]: osdmap e194: 8 total, 8 up, 8 in 2024-09-10T18:22:16.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:15 smithi099 ceph-mon[27037]: pgmap v251: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:16.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:15 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:16.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:15 smithi099 ceph-mon[27037]: from='client.34940 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:16.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:15 smithi099 ceph-mon[27037]: osdmap e194: 8 total, 8 up, 8 in 2024-09-10T18:22:16.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:15 smithi099 ceph-mon[37557]: pgmap v251: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:16.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:15 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:16.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:15 smithi099 ceph-mon[37557]: from='client.34940 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:16.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:15 smithi099 ceph-mon[37557]: osdmap e194: 8 total, 8 up, 8 in 2024-09-10T18:22:16.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:16 smithi133 ceph-mon[31214]: osdmap e195: 8 total, 8 up, 8 in 2024-09-10T18:22:17.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:16 smithi099 ceph-mon[27037]: osdmap e195: 8 total, 8 up, 8 in 2024-09-10T18:22:17.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:16 smithi099 ceph-mon[37557]: osdmap e195: 8 total, 8 up, 8 in 2024-09-10T18:22:17.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:22:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:22:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:22:17.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:17 smithi133 ceph-mon[31214]: pgmap v254: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:17.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:17 smithi133 ceph-mon[31214]: osdmap e196: 8 total, 8 up, 8 in 2024-09-10T18:22:18.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:17 smithi099 ceph-mon[27037]: pgmap v254: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:18.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:17 smithi099 ceph-mon[27037]: osdmap e196: 8 total, 8 up, 8 in 2024-09-10T18:22:18.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:17 smithi099 ceph-mon[37557]: pgmap v254: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:18.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:17 smithi099 ceph-mon[37557]: osdmap e196: 8 total, 8 up, 8 in 2024-09-10T18:22:18.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:18 smithi133 ceph-mon[31214]: osdmap e197: 8 total, 8 up, 8 in 2024-09-10T18:22:18.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:18 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2866091855' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:18.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:18 smithi133 ceph-mon[31214]: from='client.34946 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:19.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:18 smithi099 ceph-mon[27037]: osdmap e197: 8 total, 8 up, 8 in 2024-09-10T18:22:19.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:18 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2866091855' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:19.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:18 smithi099 ceph-mon[27037]: from='client.34946 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:19.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:18 smithi099 ceph-mon[37557]: osdmap e197: 8 total, 8 up, 8 in 2024-09-10T18:22:19.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:18 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2866091855' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:19.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:18 smithi099 ceph-mon[37557]: from='client.34946 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:19.635 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_ops PASSED [ 52%] 2024-09-10T18:22:19.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:19 smithi133 ceph-mon[31214]: pgmap v257: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-10T18:22:19.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:19 smithi133 ceph-mon[31214]: from='client.34946 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:19.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:19 smithi133 ceph-mon[31214]: osdmap e198: 8 total, 8 up, 8 in 2024-09-10T18:22:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:19 smithi099 ceph-mon[27037]: pgmap v257: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-10T18:22:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:19 smithi099 ceph-mon[27037]: from='client.34946 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:19 smithi099 ceph-mon[27037]: osdmap e198: 8 total, 8 up, 8 in 2024-09-10T18:22:20.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:19 smithi099 ceph-mon[37557]: pgmap v257: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-10T18:22:20.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:19 smithi099 ceph-mon[37557]: from='client.34946 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:20.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:19 smithi099 ceph-mon[37557]: osdmap e198: 8 total, 8 up, 8 in 2024-09-10T18:22:20.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:20 smithi133 ceph-mon[31214]: osdmap e199: 8 total, 8 up, 8 in 2024-09-10T18:22:21.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:20 smithi099 ceph-mon[27037]: osdmap e199: 8 total, 8 up, 8 in 2024-09-10T18:22:21.009 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:20 smithi099 ceph-mon[37557]: osdmap e199: 8 total, 8 up, 8 in 2024-09-10T18:22:21.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:21 smithi133 ceph-mon[31214]: pgmap v260: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:21.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:21 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:21.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:21 smithi133 ceph-mon[31214]: osdmap e200: 8 total, 8 up, 8 in 2024-09-10T18:22:22.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:21 smithi099 ceph-mon[27037]: pgmap v260: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:22.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:21 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:22.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:21 smithi099 ceph-mon[27037]: osdmap e200: 8 total, 8 up, 8 in 2024-09-10T18:22:22.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:21 smithi099 ceph-mon[37557]: pgmap v260: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:22.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:21 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:22.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:21 smithi099 ceph-mon[37557]: osdmap e200: 8 total, 8 up, 8 in 2024-09-10T18:22:22.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:22 smithi133 ceph-mon[31214]: osdmap e201: 8 total, 8 up, 8 in 2024-09-10T18:22:22.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:22 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/4025507244' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:22.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:22 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:23.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:22 smithi099 ceph-mon[27037]: osdmap e201: 8 total, 8 up, 8 in 2024-09-10T18:22:23.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:22 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/4025507244' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:23.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:22 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:23.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:22 smithi099 ceph-mon[37557]: osdmap e201: 8 total, 8 up, 8 in 2024-09-10T18:22:23.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:22 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/4025507244' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:23.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:22 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:23.670 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_execute_op PASSED [ 53%] 2024-09-10T18:22:23.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:22:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:22:23.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:23 smithi133 ceph-mon[31214]: pgmap v263: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:23.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:23 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/4025507244' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:23.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:23 smithi133 ceph-mon[31214]: osdmap e202: 8 total, 8 up, 8 in 2024-09-10T18:22:24.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:23 smithi099 ceph-mon[27037]: pgmap v263: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:24.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:23 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/4025507244' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:24.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:23 smithi099 ceph-mon[27037]: osdmap e202: 8 total, 8 up, 8 in 2024-09-10T18:22:24.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:22:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:22:23] "GET /metrics HTTP/1.1" 200 225016 "" "Prometheus/2.43.0" 2024-09-10T18:22:24.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:23 smithi099 ceph-mon[37557]: pgmap v263: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:24.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:23 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/4025507244' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:24.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:23 smithi099 ceph-mon[37557]: osdmap e202: 8 total, 8 up, 8 in 2024-09-10T18:22:24.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:24 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:24.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:24 smithi133 ceph-mon[31214]: osdmap e203: 8 total, 8 up, 8 in 2024-09-10T18:22:25.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:24 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:25.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:24 smithi099 ceph-mon[27037]: osdmap e203: 8 total, 8 up, 8 in 2024-09-10T18:22:25.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:24 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:25.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:24 smithi099 ceph-mon[37557]: osdmap e203: 8 total, 8 up, 8 in 2024-09-10T18:22:25.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:25 smithi133 ceph-mon[31214]: pgmap v266: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:25.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:25 smithi133 ceph-mon[31214]: osdmap e204: 8 total, 8 up, 8 in 2024-09-10T18:22:26.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:25 smithi099 ceph-mon[27037]: pgmap v266: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:26.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:25 smithi099 ceph-mon[27037]: osdmap e204: 8 total, 8 up, 8 in 2024-09-10T18:22:26.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:25 smithi099 ceph-mon[37557]: pgmap v266: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:26.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:25 smithi099 ceph-mon[37557]: osdmap e204: 8 total, 8 up, 8 in 2024-09-10T18:22:26.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:26 smithi133 ceph-mon[31214]: osdmap e205: 8 total, 8 up, 8 in 2024-09-10T18:22:26.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:26 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1548466939' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:27.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:26 smithi099 ceph-mon[27037]: osdmap e205: 8 total, 8 up, 8 in 2024-09-10T18:22:27.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:26 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1548466939' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:27.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:26 smithi099 ceph-mon[37557]: osdmap e205: 8 total, 8 up, 8 in 2024-09-10T18:22:27.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:26 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1548466939' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:27.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:22:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:22:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:22:27.702 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_writesame_op PASSED [ 55%] 2024-09-10T18:22:27.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:27 smithi133 ceph-mon[31214]: pgmap v269: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:27.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:27 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:27.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:27 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1548466939' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:27.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:27 smithi133 ceph-mon[31214]: osdmap e206: 8 total, 8 up, 8 in 2024-09-10T18:22:28.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:27 smithi099 ceph-mon[37557]: pgmap v269: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:28.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:27 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:28.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:27 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1548466939' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:28.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:27 smithi099 ceph-mon[37557]: osdmap e206: 8 total, 8 up, 8 in 2024-09-10T18:22:28.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:27 smithi099 ceph-mon[27037]: pgmap v269: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:28.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:27 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:28.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:27 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1548466939' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:28.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:27 smithi099 ceph-mon[27037]: osdmap e206: 8 total, 8 up, 8 in 2024-09-10T18:22:28.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:28 smithi133 ceph-mon[31214]: osdmap e207: 8 total, 8 up, 8 in 2024-09-10T18:22:29.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:28 smithi099 ceph-mon[27037]: osdmap e207: 8 total, 8 up, 8 in 2024-09-10T18:22:29.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:28 smithi099 ceph-mon[37557]: osdmap e207: 8 total, 8 up, 8 in 2024-09-10T18:22:29.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:29 smithi133 ceph-mon[31214]: pgmap v272: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:29 smithi133 ceph-mon[31214]: osdmap e208: 8 total, 8 up, 8 in 2024-09-10T18:22:30.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:29 smithi099 ceph-mon[27037]: pgmap v272: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:30.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:29 smithi099 ceph-mon[27037]: osdmap e208: 8 total, 8 up, 8 in 2024-09-10T18:22:30.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:29 smithi099 ceph-mon[37557]: pgmap v272: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:30.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:29 smithi099 ceph-mon[37557]: osdmap e208: 8 total, 8 up, 8 in 2024-09-10T18:22:30.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:30 smithi133 ceph-mon[31214]: osdmap e209: 8 total, 8 up, 8 in 2024-09-10T18:22:30.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:30 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3929458817' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:31.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:30 smithi099 ceph-mon[27037]: osdmap e209: 8 total, 8 up, 8 in 2024-09-10T18:22:31.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:30 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3929458817' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:31.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:30 smithi099 ceph-mon[37557]: osdmap e209: 8 total, 8 up, 8 in 2024-09-10T18:22:31.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:30 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3929458817' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:31.738 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_omap_vals_by_keys PASSED [ 56%] 2024-09-10T18:22:32.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:31 smithi099 ceph-mon[27037]: pgmap v275: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:32.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:31 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3929458817' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:32.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:31 smithi099 ceph-mon[27037]: osdmap e210: 8 total, 8 up, 8 in 2024-09-10T18:22:32.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:31 smithi099 ceph-mon[37557]: pgmap v275: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:32.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:31 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3929458817' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:32.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:31 smithi099 ceph-mon[37557]: osdmap e210: 8 total, 8 up, 8 in 2024-09-10T18:22:32.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:31 smithi133 ceph-mon[31214]: pgmap v275: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:32.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:31 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3929458817' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:32.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:31 smithi133 ceph-mon[31214]: osdmap e210: 8 total, 8 up, 8 in 2024-09-10T18:22:33.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:32 smithi099 ceph-mon[27037]: osdmap e211: 8 total, 8 up, 8 in 2024-09-10T18:22:33.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:32 smithi099 ceph-mon[37557]: osdmap e211: 8 total, 8 up, 8 in 2024-09-10T18:22:33.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:32 smithi133 ceph-mon[31214]: osdmap e211: 8 total, 8 up, 8 in 2024-09-10T18:22:33.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:33 smithi133 ceph-mon[31214]: pgmap v278: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:33.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:33 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:33.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:33 smithi133 ceph-mon[31214]: osdmap e212: 8 total, 8 up, 8 in 2024-09-10T18:22:33.991 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:22:33 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:22:34.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:33 smithi099 ceph-mon[27037]: pgmap v278: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:34.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:33 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:34.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:33 smithi099 ceph-mon[27037]: osdmap e212: 8 total, 8 up, 8 in 2024-09-10T18:22:34.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:22:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:22:33] "GET /metrics HTTP/1.1" 200 225016 "" "Prometheus/2.43.0" 2024-09-10T18:22:34.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:33 smithi099 ceph-mon[37557]: pgmap v278: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:34.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:33 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:34.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:33 smithi099 ceph-mon[37557]: osdmap e212: 8 total, 8 up, 8 in 2024-09-10T18:22:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:34 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:34 smithi099 ceph-mon[27037]: osdmap e213: 8 total, 8 up, 8 in 2024-09-10T18:22:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:34 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3282186069' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:34 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:34 smithi099 ceph-mon[37557]: osdmap e213: 8 total, 8 up, 8 in 2024-09-10T18:22:35.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:34 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3282186069' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:35.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:34 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:35.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:34 smithi133 ceph-mon[31214]: osdmap e213: 8 total, 8 up, 8 in 2024-09-10T18:22:35.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:34 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3282186069' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:35.769 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_omap_keys PASSED [ 57%] 2024-09-10T18:22:36.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:35 smithi133 ceph-mon[31214]: pgmap v281: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:36.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:35 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3282186069' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:36.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:35 smithi133 ceph-mon[31214]: osdmap e214: 8 total, 8 up, 8 in 2024-09-10T18:22:36.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:35 smithi099 ceph-mon[27037]: pgmap v281: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:36.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:35 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3282186069' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:36.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:35 smithi099 ceph-mon[27037]: osdmap e214: 8 total, 8 up, 8 in 2024-09-10T18:22:36.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:35 smithi099 ceph-mon[37557]: pgmap v281: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:36.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:35 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3282186069' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:36.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:35 smithi099 ceph-mon[37557]: osdmap e214: 8 total, 8 up, 8 in 2024-09-10T18:22:37.171 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:36 smithi133 ceph-mon[31214]: osdmap e215: 8 total, 8 up, 8 in 2024-09-10T18:22:37.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:36 smithi099 ceph-mon[27037]: osdmap e215: 8 total, 8 up, 8 in 2024-09-10T18:22:37.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:36 smithi099 ceph-mon[37557]: osdmap e215: 8 total, 8 up, 8 in 2024-09-10T18:22:37.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:22:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:22:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:22:38.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:37 smithi133 ceph-mon[31214]: pgmap v284: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:38.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:37 smithi133 ceph-mon[31214]: osdmap e216: 8 total, 8 up, 8 in 2024-09-10T18:22:38.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:37 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:38.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:37 smithi099 ceph-mon[27037]: pgmap v284: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:38.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:37 smithi099 ceph-mon[27037]: osdmap e216: 8 total, 8 up, 8 in 2024-09-10T18:22:38.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:37 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:38.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:37 smithi099 ceph-mon[37557]: pgmap v284: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:38.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:37 smithi099 ceph-mon[37557]: osdmap e216: 8 total, 8 up, 8 in 2024-09-10T18:22:38.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:37 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:39.156 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:38 smithi099 ceph-mon[27037]: osdmap e217: 8 total, 8 up, 8 in 2024-09-10T18:22:39.157 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:38 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1782105699' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:39.157 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:38 smithi099 ceph-mon[37557]: osdmap e217: 8 total, 8 up, 8 in 2024-09-10T18:22:39.157 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:38 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1782105699' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:39.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:38 smithi133 ceph-mon[31214]: osdmap e217: 8 total, 8 up, 8 in 2024-09-10T18:22:39.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:38 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1782105699' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:39.802 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_clear_omap PASSED [ 58%] 2024-09-10T18:22:40.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:39 smithi133 ceph-mon[31214]: pgmap v287: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:22:40.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:39 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:40.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:39 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1782105699' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:40.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:39 smithi133 ceph-mon[31214]: osdmap e218: 8 total, 8 up, 8 in 2024-09-10T18:22:40.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:39 smithi099 ceph-mon[27037]: pgmap v287: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:22:40.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:39 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:40.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:39 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1782105699' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:40.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:39 smithi099 ceph-mon[27037]: osdmap e218: 8 total, 8 up, 8 in 2024-09-10T18:22:40.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:39 smithi099 ceph-mon[37557]: pgmap v287: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:22:40.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:39 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:40.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:39 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1782105699' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:40.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:39 smithi099 ceph-mon[37557]: osdmap e218: 8 total, 8 up, 8 in 2024-09-10T18:22:41.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:40 smithi133 ceph-mon[31214]: osdmap e219: 8 total, 8 up, 8 in 2024-09-10T18:22:41.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:40 smithi099 ceph-mon[27037]: osdmap e219: 8 total, 8 up, 8 in 2024-09-10T18:22:41.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:40 smithi099 ceph-mon[37557]: osdmap e219: 8 total, 8 up, 8 in 2024-09-10T18:22:42.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:41 smithi133 ceph-mon[31214]: pgmap v290: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:42.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:41 smithi133 ceph-mon[31214]: osdmap e220: 8 total, 8 up, 8 in 2024-09-10T18:22:42.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:41 smithi099 ceph-mon[27037]: pgmap v290: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:42.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:41 smithi099 ceph-mon[27037]: osdmap e220: 8 total, 8 up, 8 in 2024-09-10T18:22:42.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:41 smithi099 ceph-mon[37557]: pgmap v290: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:42.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:41 smithi099 ceph-mon[37557]: osdmap e220: 8 total, 8 up, 8 in 2024-09-10T18:22:43.142 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:42 smithi133 ceph-mon[31214]: osdmap e221: 8 total, 8 up, 8 in 2024-09-10T18:22:43.142 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:42 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/785494387' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:43.142 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:42 smithi133 ceph-mon[31214]: from='client.34982 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:43.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:42 smithi099 ceph-mon[27037]: osdmap e221: 8 total, 8 up, 8 in 2024-09-10T18:22:43.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:42 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/785494387' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:43.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:42 smithi099 ceph-mon[27037]: from='client.34982 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:43.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:42 smithi099 ceph-mon[37557]: osdmap e221: 8 total, 8 up, 8 in 2024-09-10T18:22:43.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:42 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/785494387' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:43.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:42 smithi099 ceph-mon[37557]: from='client.34982 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:43.820 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:22:43 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:22:43.839 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_remove_omap_ramge2 PASSED [ 59%] 2024-09-10T18:22:44.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:43 smithi099 ceph-mon[27037]: pgmap v293: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:44.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:43 smithi099 ceph-mon[27037]: from='client.34982 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:44.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:43 smithi099 ceph-mon[27037]: osdmap e222: 8 total, 8 up, 8 in 2024-09-10T18:22:44.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:22:43 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:22:43] "GET /metrics HTTP/1.1" 200 225044 "" "Prometheus/2.43.0" 2024-09-10T18:22:44.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:43 smithi099 ceph-mon[37557]: pgmap v293: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:44.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:43 smithi099 ceph-mon[37557]: from='client.34982 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:44.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:43 smithi099 ceph-mon[37557]: osdmap e222: 8 total, 8 up, 8 in 2024-09-10T18:22:44.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:43 smithi133 ceph-mon[31214]: pgmap v293: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:44.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:43 smithi133 ceph-mon[31214]: from='client.34982 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:44.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:43 smithi133 ceph-mon[31214]: osdmap e222: 8 total, 8 up, 8 in 2024-09-10T18:22:45.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:44 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:45.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:44 smithi133 ceph-mon[31214]: osdmap e223: 8 total, 8 up, 8 in 2024-09-10T18:22:45.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:44 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:45.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:44 smithi099 ceph-mon[27037]: osdmap e223: 8 total, 8 up, 8 in 2024-09-10T18:22:45.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:44 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:45.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:44 smithi099 ceph-mon[37557]: osdmap e223: 8 total, 8 up, 8 in 2024-09-10T18:22:46.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:45 smithi133 ceph-mon[31214]: pgmap v296: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:46.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:45 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:46.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:45 smithi133 ceph-mon[31214]: osdmap e224: 8 total, 8 up, 8 in 2024-09-10T18:22:46.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:45 smithi099 ceph-mon[27037]: pgmap v296: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:46.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:45 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:46.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:45 smithi099 ceph-mon[27037]: osdmap e224: 8 total, 8 up, 8 in 2024-09-10T18:22:46.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:45 smithi099 ceph-mon[37557]: pgmap v296: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:46.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:45 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:46.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:45 smithi099 ceph-mon[37557]: osdmap e224: 8 total, 8 up, 8 in 2024-09-10T18:22:47.172 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:46 smithi133 ceph-mon[31214]: osdmap e225: 8 total, 8 up, 8 in 2024-09-10T18:22:47.172 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:46 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3816880530' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:47.172 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:46 smithi133 ceph-mon[31214]: from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:47.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:46 smithi099 ceph-mon[27037]: osdmap e225: 8 total, 8 up, 8 in 2024-09-10T18:22:47.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:46 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3816880530' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:47.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:46 smithi099 ceph-mon[27037]: from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:47.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:46 smithi099 ceph-mon[37557]: osdmap e225: 8 total, 8 up, 8 in 2024-09-10T18:22:47.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:46 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3816880530' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:47.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:46 smithi099 ceph-mon[37557]: from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:47.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:22:47 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:22:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:22:47.851 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_omap_cmp PASSED [ 60%] 2024-09-10T18:22:48.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:47 smithi133 ceph-mon[31214]: pgmap v299: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:48.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:47 smithi133 ceph-mon[31214]: from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:48.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:47 smithi133 ceph-mon[31214]: osdmap e226: 8 total, 8 up, 8 in 2024-09-10T18:22:48.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:47 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:22:48.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:47 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:22:48.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:47 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:22:48.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:47 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:22:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[27037]: pgmap v299: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[27037]: from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[27037]: osdmap e226: 8 total, 8 up, 8 in 2024-09-10T18:22:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:22:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:22:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:22:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:22:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[37557]: pgmap v299: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[37557]: from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[37557]: osdmap e226: 8 total, 8 up, 8 in 2024-09-10T18:22:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:22:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:22:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:22:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:47 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:22:49.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:48 smithi133 ceph-mon[31214]: osdmap e227: 8 total, 8 up, 8 in 2024-09-10T18:22:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:48 smithi099 ceph-mon[27037]: osdmap e227: 8 total, 8 up, 8 in 2024-09-10T18:22:49.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:48 smithi099 ceph-mon[37557]: osdmap e227: 8 total, 8 up, 8 in 2024-09-10T18:22:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:49 smithi133 ceph-mon[31214]: pgmap v302: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:22:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:49 smithi133 ceph-mon[31214]: osdmap e228: 8 total, 8 up, 8 in 2024-09-10T18:22:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:49 smithi099 ceph-mon[27037]: pgmap v302: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:22:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:49 smithi099 ceph-mon[27037]: osdmap e228: 8 total, 8 up, 8 in 2024-09-10T18:22:50.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:49 smithi099 ceph-mon[37557]: pgmap v302: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:22:50.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:49 smithi099 ceph-mon[37557]: osdmap e228: 8 total, 8 up, 8 in 2024-09-10T18:22:51.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:50 smithi133 ceph-mon[31214]: osdmap e229: 8 total, 8 up, 8 in 2024-09-10T18:22:51.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:50 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3228640065' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:51.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:50 smithi133 ceph-mon[31214]: from='client.34994 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:51.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:50 smithi099 ceph-mon[27037]: osdmap e229: 8 total, 8 up, 8 in 2024-09-10T18:22:51.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:50 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3228640065' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:51.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:50 smithi099 ceph-mon[27037]: from='client.34994 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:51.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:50 smithi099 ceph-mon[37557]: osdmap e229: 8 total, 8 up, 8 in 2024-09-10T18:22:51.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:50 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3228640065' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:51.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:50 smithi099 ceph-mon[37557]: from='client.34994 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:51.892 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_cmpext_op PASSED [ 61%] 2024-09-10T18:22:52.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:51 smithi133 ceph-mon[31214]: pgmap v305: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:22:52.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:51 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:52.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:51 smithi133 ceph-mon[31214]: from='client.34994 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:52.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:51 smithi133 ceph-mon[31214]: osdmap e230: 8 total, 8 up, 8 in 2024-09-10T18:22:52.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:51 smithi099 ceph-mon[27037]: pgmap v305: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:22:52.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:51 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:52.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:51 smithi099 ceph-mon[27037]: from='client.34994 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:52.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:51 smithi099 ceph-mon[27037]: osdmap e230: 8 total, 8 up, 8 in 2024-09-10T18:22:52.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:51 smithi099 ceph-mon[37557]: pgmap v305: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:22:52.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:51 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:52.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:51 smithi099 ceph-mon[37557]: from='client.34994 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:52.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:51 smithi099 ceph-mon[37557]: osdmap e230: 8 total, 8 up, 8 in 2024-09-10T18:22:53.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:52 smithi133 ceph-mon[31214]: osdmap e231: 8 total, 8 up, 8 in 2024-09-10T18:22:53.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:52 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:53.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:52 smithi099 ceph-mon[27037]: osdmap e231: 8 total, 8 up, 8 in 2024-09-10T18:22:53.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:52 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:53.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:52 smithi099 ceph-mon[37557]: osdmap e231: 8 total, 8 up, 8 in 2024-09-10T18:22:53.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:52 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:22:53.892 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:22:53 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:22:54.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:53 smithi099 ceph-mon[27037]: pgmap v308: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:54.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:53 smithi099 ceph-mon[27037]: osdmap e232: 8 total, 8 up, 8 in 2024-09-10T18:22:54.006 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:22:53 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:22:53] "GET /metrics HTTP/1.1" 200 222715 "" "Prometheus/2.43.0" 2024-09-10T18:22:54.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:53 smithi099 ceph-mon[37557]: pgmap v308: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:54.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:53 smithi099 ceph-mon[37557]: osdmap e232: 8 total, 8 up, 8 in 2024-09-10T18:22:54.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:53 smithi133 ceph-mon[31214]: pgmap v308: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:22:54.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:53 smithi133 ceph-mon[31214]: osdmap e232: 8 total, 8 up, 8 in 2024-09-10T18:22:55.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:54 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:55.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:54 smithi133 ceph-mon[31214]: osdmap e233: 8 total, 8 up, 8 in 2024-09-10T18:22:55.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:54 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/965421238' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:55.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:54 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:55.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:54 smithi099 ceph-mon[27037]: osdmap e233: 8 total, 8 up, 8 in 2024-09-10T18:22:55.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:54 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/965421238' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:55.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:54 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:22:55.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:54 smithi099 ceph-mon[37557]: osdmap e233: 8 total, 8 up, 8 in 2024-09-10T18:22:55.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:54 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/965421238' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:55.914 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_xattrs_op PASSED [ 62%] 2024-09-10T18:22:56.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:55 smithi133 ceph-mon[31214]: pgmap v311: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:56.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:55 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/965421238' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:56.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:55 smithi133 ceph-mon[31214]: osdmap e234: 8 total, 8 up, 8 in 2024-09-10T18:22:56.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:55 smithi099 ceph-mon[27037]: pgmap v311: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:56.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:55 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/965421238' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:56.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:55 smithi099 ceph-mon[27037]: osdmap e234: 8 total, 8 up, 8 in 2024-09-10T18:22:56.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:55 smithi099 ceph-mon[37557]: pgmap v311: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:56.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:55 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/965421238' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:22:56.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:55 smithi099 ceph-mon[37557]: osdmap e234: 8 total, 8 up, 8 in 2024-09-10T18:22:57.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:56 smithi133 ceph-mon[31214]: osdmap e235: 8 total, 8 up, 8 in 2024-09-10T18:22:57.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:56 smithi133 ceph-mon[31214]: pgmap v314: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:57.241 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:22:57 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:22:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:22:57.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:56 smithi099 ceph-mon[27037]: osdmap e235: 8 total, 8 up, 8 in 2024-09-10T18:22:57.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:56 smithi099 ceph-mon[27037]: pgmap v314: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:57.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:56 smithi099 ceph-mon[37557]: osdmap e235: 8 total, 8 up, 8 in 2024-09-10T18:22:57.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:56 smithi099 ceph-mon[37557]: pgmap v314: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:22:58.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:57 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:58.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:57 smithi133 ceph-mon[31214]: osdmap e236: 8 total, 8 up, 8 in 2024-09-10T18:22:58.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:57 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:58.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:57 smithi099 ceph-mon[27037]: osdmap e236: 8 total, 8 up, 8 in 2024-09-10T18:22:58.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:57 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:22:58.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:57 smithi099 ceph-mon[37557]: osdmap e236: 8 total, 8 up, 8 in 2024-09-10T18:22:59.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:58 smithi133 ceph-mon[31214]: osdmap e237: 8 total, 8 up, 8 in 2024-09-10T18:22:59.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:58 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3791822576' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:59.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:58 smithi133 ceph-mon[31214]: from='client.35006 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:59.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:58 smithi133 ceph-mon[31214]: pgmap v317: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:22:59.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:58 smithi099 ceph-mon[27037]: osdmap e237: 8 total, 8 up, 8 in 2024-09-10T18:22:59.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:58 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3791822576' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:59.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:58 smithi099 ceph-mon[27037]: from='client.35006 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:59.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:58 smithi099 ceph-mon[27037]: pgmap v317: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:22:59.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:58 smithi099 ceph-mon[37557]: osdmap e237: 8 total, 8 up, 8 in 2024-09-10T18:22:59.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:58 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3791822576' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:59.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:58 smithi099 ceph-mon[37557]: from='client.35006 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:22:59.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:58 smithi099 ceph-mon[37557]: pgmap v317: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:22:59.947 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_locator PASSED [ 64%] 2024-09-10T18:23:00.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:59 smithi133 ceph-mon[31214]: from='client.35006 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:00.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:22:59 smithi133 ceph-mon[31214]: osdmap e238: 8 total, 8 up, 8 in 2024-09-10T18:23:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:59 smithi099 ceph-mon[27037]: from='client.35006 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:22:59 smithi099 ceph-mon[27037]: osdmap e238: 8 total, 8 up, 8 in 2024-09-10T18:23:00.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:59 smithi099 ceph-mon[37557]: from='client.35006 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:00.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:22:59 smithi099 ceph-mon[37557]: osdmap e238: 8 total, 8 up, 8 in 2024-09-10T18:23:01.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:00 smithi133 ceph-mon[31214]: osdmap e239: 8 total, 8 up, 8 in 2024-09-10T18:23:01.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:00 smithi133 ceph-mon[31214]: pgmap v320: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:01.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:00 smithi099 ceph-mon[27037]: osdmap e239: 8 total, 8 up, 8 in 2024-09-10T18:23:01.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:00 smithi099 ceph-mon[27037]: pgmap v320: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:01.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:00 smithi099 ceph-mon[37557]: osdmap e239: 8 total, 8 up, 8 in 2024-09-10T18:23:01.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:00 smithi099 ceph-mon[37557]: pgmap v320: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:02.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:01 smithi133 ceph-mon[31214]: osdmap e240: 8 total, 8 up, 8 in 2024-09-10T18:23:02.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:01 smithi099 ceph-mon[37557]: osdmap e240: 8 total, 8 up, 8 in 2024-09-10T18:23:02.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:01 smithi099 ceph-mon[27037]: osdmap e240: 8 total, 8 up, 8 in 2024-09-10T18:23:03.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:02 smithi133 ceph-mon[31214]: osdmap e241: 8 total, 8 up, 8 in 2024-09-10T18:23:03.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:02 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/611100775' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:03.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:02 smithi133 ceph-mon[31214]: from='client.35012 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:03.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:02 smithi133 ceph-mon[31214]: pgmap v323: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:03.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:02 smithi099 ceph-mon[27037]: osdmap e241: 8 total, 8 up, 8 in 2024-09-10T18:23:03.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:02 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/611100775' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:03.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:02 smithi099 ceph-mon[27037]: from='client.35012 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:03.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:02 smithi099 ceph-mon[27037]: pgmap v323: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:03.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:02 smithi099 ceph-mon[37557]: osdmap e241: 8 total, 8 up, 8 in 2024-09-10T18:23:03.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:02 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/611100775' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:03.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:02 smithi099 ceph-mon[37557]: from='client.35012 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:03.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:02 smithi099 ceph-mon[37557]: pgmap v323: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:03.980 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:23:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:23:03.981 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:23:03 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:23:03] "GET /metrics HTTP/1.1" 200 222715 "" "Prometheus/2.43.0" 2024-09-10T18:23:03.982 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_operate_aio_write_op PASSED [ 65%] 2024-09-10T18:23:04.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:03 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:04.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:03 smithi133 ceph-mon[31214]: from='client.35012 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:04.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:03 smithi133 ceph-mon[31214]: osdmap e242: 8 total, 8 up, 8 in 2024-09-10T18:23:04.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:03 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:04.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:03 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:04.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:03 smithi099 ceph-mon[27037]: from='client.35012 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:04.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:03 smithi099 ceph-mon[27037]: osdmap e242: 8 total, 8 up, 8 in 2024-09-10T18:23:04.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:03 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:04.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:03 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:04.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:03 smithi099 ceph-mon[37557]: from='client.35012 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:04.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:03 smithi099 ceph-mon[37557]: osdmap e242: 8 total, 8 up, 8 in 2024-09-10T18:23:04.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:03 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:05.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:04 smithi133 ceph-mon[31214]: osdmap e243: 8 total, 8 up, 8 in 2024-09-10T18:23:05.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:04 smithi133 ceph-mon[31214]: pgmap v326: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:05.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:04 smithi099 ceph-mon[37557]: osdmap e243: 8 total, 8 up, 8 in 2024-09-10T18:23:05.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:04 smithi099 ceph-mon[37557]: pgmap v326: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:05.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:04 smithi099 ceph-mon[27037]: osdmap e243: 8 total, 8 up, 8 in 2024-09-10T18:23:05.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:04 smithi099 ceph-mon[27037]: pgmap v326: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:06.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:05 smithi099 ceph-mon[37557]: osdmap e244: 8 total, 8 up, 8 in 2024-09-10T18:23:06.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:05 smithi099 ceph-mon[27037]: osdmap e244: 8 total, 8 up, 8 in 2024-09-10T18:23:06.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:05 smithi133 ceph-mon[31214]: osdmap e244: 8 total, 8 up, 8 in 2024-09-10T18:23:07.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:07 smithi099 ceph-mon[27037]: osdmap e245: 8 total, 8 up, 8 in 2024-09-10T18:23:07.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:07 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/844040126' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:07 smithi099 ceph-mon[27037]: from='client.35018 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:07.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:07 smithi099 ceph-mon[27037]: pgmap v329: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:07.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:07 smithi099 ceph-mon[37557]: osdmap e245: 8 total, 8 up, 8 in 2024-09-10T18:23:07.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:07 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/844040126' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:07.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:07 smithi099 ceph-mon[37557]: from='client.35018 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:07.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:07 smithi099 ceph-mon[37557]: pgmap v329: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:07.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:23:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:23:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:23:07.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:07 smithi133 ceph-mon[31214]: osdmap e245: 8 total, 8 up, 8 in 2024-09-10T18:23:07.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:07 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/844040126' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:07.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:07 smithi133 ceph-mon[31214]: from='client.35018 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:07.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:07 smithi133 ceph-mon[31214]: pgmap v329: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:08.019 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write PASSED [ 66%] 2024-09-10T18:23:08.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:08 smithi133 ceph-mon[31214]: from='client.35018 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:08.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:08 smithi133 ceph-mon[31214]: osdmap e246: 8 total, 8 up, 8 in 2024-09-10T18:23:08.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:08 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:08.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:08 smithi099 ceph-mon[27037]: from='client.35018 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:08.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:08 smithi099 ceph-mon[27037]: osdmap e246: 8 total, 8 up, 8 in 2024-09-10T18:23:08.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:08 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:08.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:08 smithi099 ceph-mon[37557]: from='client.35018 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:08.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:08 smithi099 ceph-mon[37557]: osdmap e246: 8 total, 8 up, 8 in 2024-09-10T18:23:08.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:08 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:09.406 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:09 smithi099 ceph-mon[27037]: osdmap e247: 8 total, 8 up, 8 in 2024-09-10T18:23:09.407 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:09 smithi099 ceph-mon[27037]: pgmap v332: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:09.407 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:09 smithi099 ceph-mon[37557]: osdmap e247: 8 total, 8 up, 8 in 2024-09-10T18:23:09.407 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:09 smithi099 ceph-mon[37557]: pgmap v332: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:09.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:09 smithi133 ceph-mon[31214]: osdmap e247: 8 total, 8 up, 8 in 2024-09-10T18:23:09.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:09 smithi133 ceph-mon[31214]: pgmap v332: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:10.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:10 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:10.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:10 smithi133 ceph-mon[31214]: osdmap e248: 8 total, 8 up, 8 in 2024-09-10T18:23:10.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:10 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:10.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:10 smithi099 ceph-mon[27037]: osdmap e248: 8 total, 8 up, 8 in 2024-09-10T18:23:10.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:10 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:10.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:10 smithi099 ceph-mon[37557]: osdmap e248: 8 total, 8 up, 8 in 2024-09-10T18:23:11.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:11 smithi133 ceph-mon[31214]: osdmap e249: 8 total, 8 up, 8 in 2024-09-10T18:23:11.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:11 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2682131860' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:11.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:11 smithi133 ceph-mon[31214]: from='client.35024 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:11.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:11 smithi133 ceph-mon[31214]: pgmap v335: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:11.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:11 smithi099 ceph-mon[27037]: osdmap e249: 8 total, 8 up, 8 in 2024-09-10T18:23:11.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:11 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2682131860' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:11.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:11 smithi099 ceph-mon[27037]: from='client.35024 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:11.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:11 smithi099 ceph-mon[27037]: pgmap v335: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:11.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:11 smithi099 ceph-mon[37557]: osdmap e249: 8 total, 8 up, 8 in 2024-09-10T18:23:11.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:11 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2682131860' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:11.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:11 smithi099 ceph-mon[37557]: from='client.35024 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:11.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:11 smithi099 ceph-mon[37557]: pgmap v335: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:12.058 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_cmpext PASSED [ 67%] 2024-09-10T18:23:12.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:12 smithi133 ceph-mon[31214]: from='client.35024 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:12.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:12 smithi133 ceph-mon[31214]: osdmap e250: 8 total, 8 up, 8 in 2024-09-10T18:23:12.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:12 smithi099 ceph-mon[27037]: from='client.35024 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:12.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:12 smithi099 ceph-mon[27037]: osdmap e250: 8 total, 8 up, 8 in 2024-09-10T18:23:12.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:12 smithi099 ceph-mon[37557]: from='client.35024 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:12.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:12 smithi099 ceph-mon[37557]: osdmap e250: 8 total, 8 up, 8 in 2024-09-10T18:23:13.392 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:13 smithi133 ceph-mon[31214]: osdmap e251: 8 total, 8 up, 8 in 2024-09-10T18:23:13.392 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:13 smithi133 ceph-mon[31214]: pgmap v338: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:13.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:13 smithi099 ceph-mon[27037]: osdmap e251: 8 total, 8 up, 8 in 2024-09-10T18:23:13.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:13 smithi099 ceph-mon[27037]: pgmap v338: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:13.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:13 smithi099 ceph-mon[37557]: osdmap e251: 8 total, 8 up, 8 in 2024-09-10T18:23:13.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:13 smithi099 ceph-mon[37557]: pgmap v338: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:13.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:23:13 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:23:14.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:23:13 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:23:13] "GET /metrics HTTP/1.1" 200 225067 "" "Prometheus/2.43.0" 2024-09-10T18:23:14.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:14 smithi133 ceph-mon[31214]: osdmap e252: 8 total, 8 up, 8 in 2024-09-10T18:23:14.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:14 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:14.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:14 smithi099 ceph-mon[27037]: osdmap e252: 8 total, 8 up, 8 in 2024-09-10T18:23:14.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:14 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:14.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:14 smithi099 ceph-mon[37557]: osdmap e252: 8 total, 8 up, 8 in 2024-09-10T18:23:14.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:14 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:15.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:15 smithi133 ceph-mon[31214]: osdmap e253: 8 total, 8 up, 8 in 2024-09-10T18:23:15.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:15 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/338300053' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:15.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:15 smithi133 ceph-mon[31214]: from='client.35030 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:15.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:15 smithi133 ceph-mon[31214]: pgmap v341: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:15.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:15 smithi099 ceph-mon[27037]: osdmap e253: 8 total, 8 up, 8 in 2024-09-10T18:23:15.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:15 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/338300053' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:15.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:15 smithi099 ceph-mon[27037]: from='client.35030 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:15.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:15 smithi099 ceph-mon[27037]: pgmap v341: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:15.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:15 smithi099 ceph-mon[37557]: osdmap e253: 8 total, 8 up, 8 in 2024-09-10T18:23:15.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:15 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/338300053' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:15.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:15 smithi099 ceph-mon[37557]: from='client.35030 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:15.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:15 smithi099 ceph-mon[37557]: pgmap v341: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:16.074 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_rmxattr PASSED [ 68%] 2024-09-10T18:23:16.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:16 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:16.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:16 smithi133 ceph-mon[31214]: from='client.35030 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:16.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:16 smithi133 ceph-mon[31214]: osdmap e254: 8 total, 8 up, 8 in 2024-09-10T18:23:16.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:16 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:16.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:16 smithi099 ceph-mon[27037]: from='client.35030 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:16.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:16 smithi099 ceph-mon[27037]: osdmap e254: 8 total, 8 up, 8 in 2024-09-10T18:23:16.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:16 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:16.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:16 smithi099 ceph-mon[37557]: from='client.35030 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:16.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:16 smithi099 ceph-mon[37557]: osdmap e254: 8 total, 8 up, 8 in 2024-09-10T18:23:17.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:17 smithi133 ceph-mon[31214]: osdmap e255: 8 total, 8 up, 8 in 2024-09-10T18:23:17.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:17 smithi133 ceph-mon[31214]: pgmap v344: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:17.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:23:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:23:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:23:17.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:17 smithi099 ceph-mon[27037]: osdmap e255: 8 total, 8 up, 8 in 2024-09-10T18:23:17.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:17 smithi099 ceph-mon[27037]: pgmap v344: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:17.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:17 smithi099 ceph-mon[37557]: osdmap e255: 8 total, 8 up, 8 in 2024-09-10T18:23:17.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:17 smithi099 ceph-mon[37557]: pgmap v344: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:18.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:18 smithi133 ceph-mon[31214]: osdmap e256: 8 total, 8 up, 8 in 2024-09-10T18:23:18.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:18 smithi099 ceph-mon[27037]: osdmap e256: 8 total, 8 up, 8 in 2024-09-10T18:23:18.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:18 smithi099 ceph-mon[37557]: osdmap e256: 8 total, 8 up, 8 in 2024-09-10T18:23:19.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:19 smithi133 ceph-mon[31214]: osdmap e257: 8 total, 8 up, 8 in 2024-09-10T18:23:19.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:19 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/181457706' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:19.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:19 smithi133 ceph-mon[31214]: from='client.35036 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:19.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:19 smithi133 ceph-mon[31214]: pgmap v347: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:23:19.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:19 smithi099 ceph-mon[27037]: osdmap e257: 8 total, 8 up, 8 in 2024-09-10T18:23:19.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:19 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/181457706' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:19.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:19 smithi099 ceph-mon[27037]: from='client.35036 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:19.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:19 smithi099 ceph-mon[27037]: pgmap v347: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:23:19.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:19 smithi099 ceph-mon[37557]: osdmap e257: 8 total, 8 up, 8 in 2024-09-10T18:23:19.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:19 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/181457706' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:19.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:19 smithi099 ceph-mon[37557]: from='client.35036 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:19.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:19 smithi099 ceph-mon[37557]: pgmap v347: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:23:20.103 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write_no_comp_ref PASSED [ 69%] 2024-09-10T18:23:20.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:20 smithi133 ceph-mon[31214]: from='client.35036 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:20.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:20 smithi133 ceph-mon[31214]: osdmap e258: 8 total, 8 up, 8 in 2024-09-10T18:23:20.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:20 smithi099 ceph-mon[27037]: from='client.35036 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:20.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:20 smithi099 ceph-mon[27037]: osdmap e258: 8 total, 8 up, 8 in 2024-09-10T18:23:20.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:20 smithi099 ceph-mon[37557]: from='client.35036 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:20.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:20 smithi099 ceph-mon[37557]: osdmap e258: 8 total, 8 up, 8 in 2024-09-10T18:23:21.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:21 smithi133 ceph-mon[31214]: osdmap e259: 8 total, 8 up, 8 in 2024-09-10T18:23:21.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:21 smithi133 ceph-mon[31214]: pgmap v350: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:21.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:21 smithi099 ceph-mon[27037]: osdmap e259: 8 total, 8 up, 8 in 2024-09-10T18:23:21.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:21 smithi099 ceph-mon[27037]: pgmap v350: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:21.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:21 smithi099 ceph-mon[37557]: osdmap e259: 8 total, 8 up, 8 in 2024-09-10T18:23:21.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:21 smithi099 ceph-mon[37557]: pgmap v350: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:22.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:22 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:22.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:22 smithi133 ceph-mon[31214]: osdmap e260: 8 total, 8 up, 8 in 2024-09-10T18:23:22.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:22 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:22.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:22 smithi099 ceph-mon[27037]: osdmap e260: 8 total, 8 up, 8 in 2024-09-10T18:23:22.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:22 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:22.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:22 smithi099 ceph-mon[37557]: osdmap e260: 8 total, 8 up, 8 in 2024-09-10T18:23:23.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:23 smithi133 ceph-mon[31214]: osdmap e261: 8 total, 8 up, 8 in 2024-09-10T18:23:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:23 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3421571158' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:23 smithi133 ceph-mon[31214]: from='client.35042 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:23 smithi133 ceph-mon[31214]: pgmap v353: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:23 smithi099 ceph-mon[27037]: osdmap e261: 8 total, 8 up, 8 in 2024-09-10T18:23:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:23 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3421571158' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:23 smithi099 ceph-mon[27037]: from='client.35042 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:23 smithi099 ceph-mon[27037]: pgmap v353: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:23 smithi099 ceph-mon[37557]: osdmap e261: 8 total, 8 up, 8 in 2024-09-10T18:23:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:23 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3421571158' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:23 smithi099 ceph-mon[37557]: from='client.35042 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:23 smithi099 ceph-mon[37557]: pgmap v353: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:23.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:23:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:23:24.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:23:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:23:23] "GET /metrics HTTP/1.1" 200 225057 "" "Prometheus/2.43.0" 2024-09-10T18:23:24.135 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_append PASSED [ 70%] 2024-09-10T18:23:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:24 smithi133 ceph-mon[31214]: from='client.35042 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:24 smithi133 ceph-mon[31214]: osdmap e262: 8 total, 8 up, 8 in 2024-09-10T18:23:24.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:24 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:24.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:24 smithi099 ceph-mon[27037]: from='client.35042 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:24.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:24 smithi099 ceph-mon[27037]: osdmap e262: 8 total, 8 up, 8 in 2024-09-10T18:23:24.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:24 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:24.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:24 smithi099 ceph-mon[37557]: from='client.35042 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:24.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:24 smithi099 ceph-mon[37557]: osdmap e262: 8 total, 8 up, 8 in 2024-09-10T18:23:24.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:24 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:25.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:25 smithi133 ceph-mon[31214]: osdmap e263: 8 total, 8 up, 8 in 2024-09-10T18:23:25.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:25 smithi133 ceph-mon[31214]: pgmap v356: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:25.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:25 smithi099 ceph-mon[27037]: osdmap e263: 8 total, 8 up, 8 in 2024-09-10T18:23:25.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:25 smithi099 ceph-mon[27037]: pgmap v356: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:25.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:25 smithi099 ceph-mon[37557]: osdmap e263: 8 total, 8 up, 8 in 2024-09-10T18:23:25.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:25 smithi099 ceph-mon[37557]: pgmap v356: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:26.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:26 smithi133 ceph-mon[31214]: osdmap e264: 8 total, 8 up, 8 in 2024-09-10T18:23:26.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:26 smithi099 ceph-mon[27037]: osdmap e264: 8 total, 8 up, 8 in 2024-09-10T18:23:26.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:26 smithi099 ceph-mon[37557]: osdmap e264: 8 total, 8 up, 8 in 2024-09-10T18:23:27.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:23:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:23:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:23:27.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:27 smithi133 ceph-mon[31214]: osdmap e265: 8 total, 8 up, 8 in 2024-09-10T18:23:27.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:27 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3295426155' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:27.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:27 smithi133 ceph-mon[31214]: from='client.35048 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:27.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:27 smithi133 ceph-mon[31214]: pgmap v359: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:27.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:27 smithi099 ceph-mon[27037]: osdmap e265: 8 total, 8 up, 8 in 2024-09-10T18:23:27.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:27 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3295426155' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:27.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:27 smithi099 ceph-mon[27037]: from='client.35048 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:27.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:27 smithi099 ceph-mon[27037]: pgmap v359: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:27.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:27 smithi099 ceph-mon[37557]: osdmap e265: 8 total, 8 up, 8 in 2024-09-10T18:23:27.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:27 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3295426155' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:27.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:27 smithi099 ceph-mon[37557]: from='client.35048 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:27.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:27 smithi099 ceph-mon[37557]: pgmap v359: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:28.169 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write_full PASSED [ 71%] 2024-09-10T18:23:28.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:28 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:28.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:28 smithi133 ceph-mon[31214]: from='client.35048 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:28.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:28 smithi133 ceph-mon[31214]: osdmap e266: 8 total, 8 up, 8 in 2024-09-10T18:23:28.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:28 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:28.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:28 smithi099 ceph-mon[27037]: from='client.35048 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:28.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:28 smithi099 ceph-mon[27037]: osdmap e266: 8 total, 8 up, 8 in 2024-09-10T18:23:28.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:28 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:28.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:28 smithi099 ceph-mon[37557]: from='client.35048 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:28.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:28 smithi099 ceph-mon[37557]: osdmap e266: 8 total, 8 up, 8 in 2024-09-10T18:23:29.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:29 smithi133 ceph-mon[31214]: osdmap e267: 8 total, 8 up, 8 in 2024-09-10T18:23:29.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:29 smithi133 ceph-mon[31214]: pgmap v362: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:29.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:29 smithi099 ceph-mon[27037]: osdmap e267: 8 total, 8 up, 8 in 2024-09-10T18:23:29.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:29 smithi099 ceph-mon[27037]: pgmap v362: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:29.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:29 smithi099 ceph-mon[37557]: osdmap e267: 8 total, 8 up, 8 in 2024-09-10T18:23:29.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:29 smithi099 ceph-mon[37557]: pgmap v362: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:30.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:30 smithi133 ceph-mon[31214]: osdmap e268: 8 total, 8 up, 8 in 2024-09-10T18:23:30.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:30 smithi099 ceph-mon[27037]: osdmap e268: 8 total, 8 up, 8 in 2024-09-10T18:23:30.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:30 smithi099 ceph-mon[37557]: osdmap e268: 8 total, 8 up, 8 in 2024-09-10T18:23:31.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:31 smithi133 ceph-mon[31214]: osdmap e269: 8 total, 8 up, 8 in 2024-09-10T18:23:31.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:31 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/673269385' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:31.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:31 smithi133 ceph-mon[31214]: from='client.35054 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:31.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:31 smithi133 ceph-mon[31214]: pgmap v365: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:31.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:31 smithi099 ceph-mon[27037]: osdmap e269: 8 total, 8 up, 8 in 2024-09-10T18:23:31.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:31 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/673269385' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:31.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:31 smithi099 ceph-mon[27037]: from='client.35054 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:31.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:31 smithi099 ceph-mon[27037]: pgmap v365: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:31.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:31 smithi099 ceph-mon[37557]: osdmap e269: 8 total, 8 up, 8 in 2024-09-10T18:23:31.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:31 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/673269385' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:31.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:31 smithi099 ceph-mon[37557]: from='client.35054 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:31.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:31 smithi099 ceph-mon[37557]: pgmap v365: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:32.206 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_writesame PASSED [ 73%] 2024-09-10T18:23:32.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:32 smithi133 ceph-mon[31214]: from='client.35054 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:32.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:32 smithi133 ceph-mon[31214]: osdmap e270: 8 total, 8 up, 8 in 2024-09-10T18:23:32.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:32 smithi099 ceph-mon[27037]: from='client.35054 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:32.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:32 smithi099 ceph-mon[27037]: osdmap e270: 8 total, 8 up, 8 in 2024-09-10T18:23:32.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:32 smithi099 ceph-mon[37557]: from='client.35054 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:32.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:32 smithi099 ceph-mon[37557]: osdmap e270: 8 total, 8 up, 8 in 2024-09-10T18:23:33.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:33 smithi133 ceph-mon[31214]: osdmap e271: 8 total, 8 up, 8 in 2024-09-10T18:23:33.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:33 smithi133 ceph-mon[31214]: pgmap v368: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:33.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:33 smithi099 ceph-mon[27037]: osdmap e271: 8 total, 8 up, 8 in 2024-09-10T18:23:33.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:33 smithi099 ceph-mon[27037]: pgmap v368: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:33.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:33 smithi099 ceph-mon[37557]: osdmap e271: 8 total, 8 up, 8 in 2024-09-10T18:23:33.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:33 smithi099 ceph-mon[37557]: pgmap v368: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:33.991 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:23:33 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:23:34.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:23:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:23:33] "GET /metrics HTTP/1.1" 200 225057 "" "Prometheus/2.43.0" 2024-09-10T18:23:34.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:34 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:34.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:34 smithi133 ceph-mon[31214]: osdmap e272: 8 total, 8 up, 8 in 2024-09-10T18:23:34.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:34 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:34.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:34 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:34.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:34 smithi099 ceph-mon[27037]: osdmap e272: 8 total, 8 up, 8 in 2024-09-10T18:23:34.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:34 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:34.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:34 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:34.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:34 smithi099 ceph-mon[37557]: osdmap e272: 8 total, 8 up, 8 in 2024-09-10T18:23:34.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:34 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:35.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:35 smithi133 ceph-mon[31214]: osdmap e273: 8 total, 8 up, 8 in 2024-09-10T18:23:35.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:35 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3551476662' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:35.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:35 smithi133 ceph-mon[31214]: pgmap v371: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:35.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:35 smithi099 ceph-mon[27037]: osdmap e273: 8 total, 8 up, 8 in 2024-09-10T18:23:35.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:35 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3551476662' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:35.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:35 smithi099 ceph-mon[27037]: pgmap v371: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:35.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:35 smithi099 ceph-mon[37557]: osdmap e273: 8 total, 8 up, 8 in 2024-09-10T18:23:35.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:35 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3551476662' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:35.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:35 smithi099 ceph-mon[37557]: pgmap v371: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:36.235 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_stat PASSED [ 74%] 2024-09-10T18:23:36.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:36 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3551476662' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:36.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:36 smithi133 ceph-mon[31214]: osdmap e274: 8 total, 8 up, 8 in 2024-09-10T18:23:36.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:36 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3551476662' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:36.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:36 smithi099 ceph-mon[27037]: osdmap e274: 8 total, 8 up, 8 in 2024-09-10T18:23:36.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:36 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3551476662' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:36.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:36 smithi099 ceph-mon[37557]: osdmap e274: 8 total, 8 up, 8 in 2024-09-10T18:23:37.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:37 smithi133 ceph-mon[31214]: osdmap e275: 8 total, 8 up, 8 in 2024-09-10T18:23:37.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:37 smithi133 ceph-mon[31214]: pgmap v374: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:37.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:23:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:23:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:23:37.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:37 smithi099 ceph-mon[27037]: osdmap e275: 8 total, 8 up, 8 in 2024-09-10T18:23:37.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:37 smithi099 ceph-mon[27037]: pgmap v374: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:37.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:37 smithi099 ceph-mon[37557]: osdmap e275: 8 total, 8 up, 8 in 2024-09-10T18:23:37.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:37 smithi099 ceph-mon[37557]: pgmap v374: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:38.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:38 smithi133 ceph-mon[31214]: osdmap e276: 8 total, 8 up, 8 in 2024-09-10T18:23:38.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:38 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:38.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:38 smithi099 ceph-mon[27037]: osdmap e276: 8 total, 8 up, 8 in 2024-09-10T18:23:38.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:38 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:38.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:38 smithi099 ceph-mon[37557]: osdmap e276: 8 total, 8 up, 8 in 2024-09-10T18:23:38.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:38 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:39.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:39 smithi099 ceph-mon[27037]: osdmap e277: 8 total, 8 up, 8 in 2024-09-10T18:23:39.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:39 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/234003674' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:39.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:39 smithi099 ceph-mon[27037]: pgmap v377: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:23:39.657 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:39 smithi099 ceph-mon[37557]: osdmap e277: 8 total, 8 up, 8 in 2024-09-10T18:23:39.657 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:39 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/234003674' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:39.657 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:39 smithi099 ceph-mon[37557]: pgmap v377: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:23:39.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:39 smithi133 ceph-mon[31214]: osdmap e277: 8 total, 8 up, 8 in 2024-09-10T18:23:39.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:39 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/234003674' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:39.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:39 smithi133 ceph-mon[31214]: pgmap v377: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:23:40.277 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_remove PASSED [ 75%] 2024-09-10T18:23:40.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:40 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:40.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:40 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/234003674' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:40.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:40 smithi133 ceph-mon[31214]: osdmap e278: 8 total, 8 up, 8 in 2024-09-10T18:23:40.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:40 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:40.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:40 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/234003674' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:40.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:40 smithi099 ceph-mon[27037]: osdmap e278: 8 total, 8 up, 8 in 2024-09-10T18:23:40.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:40 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:40.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:40 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/234003674' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:40.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:40 smithi099 ceph-mon[37557]: osdmap e278: 8 total, 8 up, 8 in 2024-09-10T18:23:41.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:41 smithi133 ceph-mon[31214]: osdmap e279: 8 total, 8 up, 8 in 2024-09-10T18:23:41.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:41 smithi133 ceph-mon[31214]: pgmap v380: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:41.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:41 smithi099 ceph-mon[27037]: osdmap e279: 8 total, 8 up, 8 in 2024-09-10T18:23:41.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:41 smithi099 ceph-mon[27037]: pgmap v380: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:41.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:41 smithi099 ceph-mon[37557]: osdmap e279: 8 total, 8 up, 8 in 2024-09-10T18:23:41.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:41 smithi099 ceph-mon[37557]: pgmap v380: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:23:42.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:42 smithi133 ceph-mon[31214]: osdmap e280: 8 total, 8 up, 8 in 2024-09-10T18:23:42.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:42 smithi099 ceph-mon[27037]: osdmap e280: 8 total, 8 up, 8 in 2024-09-10T18:23:42.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:42 smithi099 ceph-mon[37557]: osdmap e280: 8 total, 8 up, 8 in 2024-09-10T18:23:43.610 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:43 smithi133 ceph-mon[31214]: osdmap e281: 8 total, 8 up, 8 in 2024-09-10T18:23:43.610 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:43 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-09-10T18:23:43.610 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:43 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:23:43.610 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:43 smithi133 ceph-mon[31214]: pgmap v383: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:43.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:43 smithi099 ceph-mon[27037]: osdmap e281: 8 total, 8 up, 8 in 2024-09-10T18:23:43.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:43 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-09-10T18:23:43.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:43 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:23:43.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:43 smithi099 ceph-mon[27037]: pgmap v383: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:43.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:43 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a[27033]: 2024-09-10T18:23:43.303+0000 7fe73a4f0640 -1 mon.a@0(leader).osd e282 definitely_dead 0 2024-09-10T18:23:43.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:43 smithi099 ceph-mon[37557]: osdmap e281: 8 total, 8 up, 8 in 2024-09-10T18:23:43.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:43 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-09-10T18:23:43.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:43 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:23:43.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:43 smithi099 ceph-mon[37557]: pgmap v383: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:23:43.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:23:43 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:23:44.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:23:43 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:23:43] "GET /metrics HTTP/1.1" 200 225078 "" "Prometheus/2.43.0" 2024-09-10T18:23:44.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:44 smithi133 ceph-mon[31214]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:23:44.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:44 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-10T18:23:44.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:44 smithi133 ceph-mon[31214]: osdmap e282: 8 total, 8 up, 8 in 2024-09-10T18:23:44.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:44 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["6", "1", "2"]}]: dispatch 2024-09-10T18:23:44.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:44 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:44.755 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:23:44 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1[47455]: 2024-09-10T18:23:44.539+0000 7ff439d9c640 -1 osd.1 283 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:23:44.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:44 smithi099 ceph-mon[27037]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:23:44.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:44 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-10T18:23:44.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:44 smithi099 ceph-mon[27037]: osdmap e282: 8 total, 8 up, 8 in 2024-09-10T18:23:44.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:44 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["6", "1", "2"]}]: dispatch 2024-09-10T18:23:44.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:44 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:44.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:44 smithi099 ceph-mon[37557]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:23:44.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:44 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-10T18:23:44.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:44 smithi099 ceph-mon[37557]: osdmap e282: 8 total, 8 up, 8 in 2024-09-10T18:23:44.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:44 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["6", "1", "2"]}]: dispatch 2024-09-10T18:23:44.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:44 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:45.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:45 smithi133 ceph-mon[31214]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-10T18:23:45.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:45 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["6", "1", "2"]}]': finished 2024-09-10T18:23:45.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:45 smithi133 ceph-mon[31214]: osdmap e283: 8 total, 5 up, 8 in 2024-09-10T18:23:45.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:45 smithi133 ceph-mon[31214]: osd.1 marked itself dead as of e283 2024-09-10T18:23:45.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:45 smithi133 ceph-mon[31214]: pgmap v386: 196 pgs: 43 stale+active+clean, 153 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:23:45.741 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:23:45 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:23:45.428+0000 7f48dfc89640 -1 osd.6 284 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:23:45.755 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:23:45 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1[47455]: 2024-09-10T18:23:45.315+0000 7ff42b391640 -1 osd.1 284 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:23:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:45 smithi099 ceph-mon[27037]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-10T18:23:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:45 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["6", "1", "2"]}]': finished 2024-09-10T18:23:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:45 smithi099 ceph-mon[27037]: osdmap e283: 8 total, 5 up, 8 in 2024-09-10T18:23:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:45 smithi099 ceph-mon[27037]: osd.1 marked itself dead as of e283 2024-09-10T18:23:45.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:45 smithi099 ceph-mon[27037]: pgmap v386: 196 pgs: 43 stale+active+clean, 153 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:23:45.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:45 smithi099 ceph-mon[37557]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-10T18:23:45.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:45 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["6", "1", "2"]}]': finished 2024-09-10T18:23:45.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:45 smithi099 ceph-mon[37557]: osdmap e283: 8 total, 5 up, 8 in 2024-09-10T18:23:45.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:45 smithi099 ceph-mon[37557]: osd.1 marked itself dead as of e283 2024-09-10T18:23:45.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:45 smithi099 ceph-mon[37557]: pgmap v386: 196 pgs: 43 stale+active+clean, 153 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:23:46.740 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:23:46 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:23:46.332+0000 7f48d2aa3640 -1 osd.6 285 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:23:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:46 smithi133 ceph-mon[31214]: Monitor daemon marked osd.1 down, but it is still running 2024-09-10T18:23:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:46 smithi133 ceph-mon[31214]: map e283 wrongly marked me down at e283 2024-09-10T18:23:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:46 smithi133 ceph-mon[31214]: osdmap e284: 8 total, 5 up, 8 in 2024-09-10T18:23:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:46 smithi133 ceph-mon[31214]: Monitor daemon marked osd.6 down, but it is still running 2024-09-10T18:23:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:46 smithi133 ceph-mon[31214]: map e284 wrongly marked me down at e283 2024-09-10T18:23:46.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:46 smithi133 ceph-mon[31214]: osd.6 marked itself dead as of e284 2024-09-10T18:23:46.755 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:23:46 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1[47455]: 2024-09-10T18:23:46.332+0000 7ff42b391640 -1 osd.1 285 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:23:46.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:46 smithi099 ceph-mon[27037]: Monitor daemon marked osd.1 down, but it is still running 2024-09-10T18:23:46.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:46 smithi099 ceph-mon[27037]: map e283 wrongly marked me down at e283 2024-09-10T18:23:46.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:46 smithi099 ceph-mon[27037]: osdmap e284: 8 total, 5 up, 8 in 2024-09-10T18:23:46.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:46 smithi099 ceph-mon[27037]: Monitor daemon marked osd.6 down, but it is still running 2024-09-10T18:23:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:46 smithi099 ceph-mon[27037]: map e284 wrongly marked me down at e283 2024-09-10T18:23:46.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:46 smithi099 ceph-mon[27037]: osd.6 marked itself dead as of e284 2024-09-10T18:23:46.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:46 smithi099 ceph-mon[37557]: Monitor daemon marked osd.1 down, but it is still running 2024-09-10T18:23:46.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:46 smithi099 ceph-mon[37557]: map e283 wrongly marked me down at e283 2024-09-10T18:23:46.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:46 smithi099 ceph-mon[37557]: osdmap e284: 8 total, 5 up, 8 in 2024-09-10T18:23:46.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:46 smithi099 ceph-mon[37557]: Monitor daemon marked osd.6 down, but it is still running 2024-09-10T18:23:46.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:46 smithi099 ceph-mon[37557]: map e284 wrongly marked me down at e283 2024-09-10T18:23:46.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:46 smithi099 ceph-mon[37557]: osd.6 marked itself dead as of e284 2024-09-10T18:23:47.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:23:47 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:23:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:23:47.491 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:23:47 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:23:47.341+0000 7f48db2c1640 -1 osd.6 286 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:23:47.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:47 smithi133 ceph-mon[31214]: osdmap e285: 8 total, 5 up, 8 in 2024-09-10T18:23:47.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:47 smithi133 ceph-mon[31214]: pgmap v389: 196 pgs: 53 stale+active+clean, 143 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:23:47.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:47 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:47.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:47 smithi099 ceph-mon[27037]: osdmap e285: 8 total, 5 up, 8 in 2024-09-10T18:23:47.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:47 smithi099 ceph-mon[27037]: pgmap v389: 196 pgs: 53 stale+active+clean, 143 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:23:47.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:47 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:47.755 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:23:47 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1[47455]: 2024-09-10T18:23:47.345+0000 7ff433baf640 -1 osd.1 286 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:23:47.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:47 smithi099 ceph-mon[37557]: osdmap e285: 8 total, 5 up, 8 in 2024-09-10T18:23:47.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:47 smithi099 ceph-mon[37557]: pgmap v389: 196 pgs: 53 stale+active+clean, 143 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:23:47.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:47 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:48.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:48 smithi133 ceph-mon[31214]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-10T18:23:48.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:48 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:48.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:48 smithi133 ceph-mon[31214]: osdmap e286: 8 total, 5 up, 8 in 2024-09-10T18:23:48.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:48 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:48.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:48 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:23:48.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:48 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:23:48.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:48 smithi133 ceph-mon[31214]: osd.2 marked itself dead as of e286 2024-09-10T18:23:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[27037]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-10T18:23:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[27037]: osdmap e286: 8 total, 5 up, 8 in 2024-09-10T18:23:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:23:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:23:48.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[27037]: osd.2 marked itself dead as of e286 2024-09-10T18:23:48.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[37557]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-10T18:23:48.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:48.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[37557]: osdmap e286: 8 total, 5 up, 8 in 2024-09-10T18:23:48.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:48.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:23:48.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:23:48.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-mon[37557]: osd.2 marked itself dead as of e286 2024-09-10T18:23:49.255 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:23:48 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2[53449]: 2024-09-10T18:23:48.983+0000 7effd4b1c640 -1 osd.2 287 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:23:49.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:49 smithi133 ceph-mon[31214]: Monitor daemon marked osd.2 down, but it is still running 2024-09-10T18:23:49.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:49 smithi133 ceph-mon[31214]: map e286 wrongly marked me down at e283 2024-09-10T18:23:49.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:49 smithi133 ceph-mon[31214]: osd.6 v1:172.21.15.133:6808/2733605656 boot 2024-09-10T18:23:49.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:49 smithi133 ceph-mon[31214]: osd.1 v1:172.21.15.99:6805/719640887 boot 2024-09-10T18:23:49.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:49 smithi133 ceph-mon[31214]: osdmap e287: 8 total, 7 up, 8 in 2024-09-10T18:23:49.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:49 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:23:49.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:49 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:23:49.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:49 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:23:49.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:49 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:23:49.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:49 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:23:49.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:49 smithi133 ceph-mon[31214]: pgmap v392: 196 pgs: 1 unknown, 1 undersized+peered, 46 active+undersized, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 64 peering, 5 stale+active+clean, 7 undersized+peered+wait, 20 active+undersized+wait, 1 undersized+degraded+peered, 14 active+undersized+degraded, 31 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 81/591 objects degraded (13.706%) 2024-09-10T18:23:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[27037]: Monitor daemon marked osd.2 down, but it is still running 2024-09-10T18:23:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[27037]: map e286 wrongly marked me down at e283 2024-09-10T18:23:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[27037]: osd.6 v1:172.21.15.133:6808/2733605656 boot 2024-09-10T18:23:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[27037]: osd.1 v1:172.21.15.99:6805/719640887 boot 2024-09-10T18:23:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[27037]: osdmap e287: 8 total, 7 up, 8 in 2024-09-10T18:23:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:23:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:23:49.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:23:49.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:23:49.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:23:49.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[27037]: pgmap v392: 196 pgs: 1 unknown, 1 undersized+peered, 46 active+undersized, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 64 peering, 5 stale+active+clean, 7 undersized+peered+wait, 20 active+undersized+wait, 1 undersized+degraded+peered, 14 active+undersized+degraded, 31 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 81/591 objects degraded (13.706%) 2024-09-10T18:23:49.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[37557]: Monitor daemon marked osd.2 down, but it is still running 2024-09-10T18:23:49.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[37557]: map e286 wrongly marked me down at e283 2024-09-10T18:23:49.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[37557]: osd.6 v1:172.21.15.133:6808/2733605656 boot 2024-09-10T18:23:49.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[37557]: osd.1 v1:172.21.15.99:6805/719640887 boot 2024-09-10T18:23:49.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[37557]: osdmap e287: 8 total, 7 up, 8 in 2024-09-10T18:23:49.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:23:49.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:23:49.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:23:49.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:23:49.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:23:49.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:49 smithi099 ceph-mon[37557]: pgmap v392: 196 pgs: 1 unknown, 1 undersized+peered, 46 active+undersized, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 64 peering, 5 stale+active+clean, 7 undersized+peered+wait, 20 active+undersized+wait, 1 undersized+degraded+peered, 14 active+undersized+degraded, 31 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 81/591 objects degraded (13.706%) 2024-09-10T18:23:50.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:50 smithi133 ceph-mon[31214]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T18:23:50.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:50 smithi133 ceph-mon[31214]: osd.2 v1:172.21.15.99:6809/592364138 boot 2024-09-10T18:23:50.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:50 smithi133 ceph-mon[31214]: osdmap e288: 8 total, 8 up, 8 in 2024-09-10T18:23:50.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:50 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:23:50.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:50 smithi133 ceph-mon[31214]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-10T18:23:50.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:50 smithi133 ceph-mon[31214]: Health check failed: Degraded data redundancy: 81/591 objects degraded (13.706%), 21 pgs degraded (PG_DEGRADED) 2024-09-10T18:23:50.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:50 smithi099 ceph-mon[27037]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T18:23:50.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:50 smithi099 ceph-mon[27037]: osd.2 v1:172.21.15.99:6809/592364138 boot 2024-09-10T18:23:50.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:50 smithi099 ceph-mon[27037]: osdmap e288: 8 total, 8 up, 8 in 2024-09-10T18:23:50.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:50 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:23:50.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:50 smithi099 ceph-mon[27037]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-10T18:23:50.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:50 smithi099 ceph-mon[27037]: Health check failed: Degraded data redundancy: 81/591 objects degraded (13.706%), 21 pgs degraded (PG_DEGRADED) 2024-09-10T18:23:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:50 smithi099 ceph-mon[37557]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T18:23:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:50 smithi099 ceph-mon[37557]: osd.2 v1:172.21.15.99:6809/592364138 boot 2024-09-10T18:23:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:50 smithi099 ceph-mon[37557]: osdmap e288: 8 total, 8 up, 8 in 2024-09-10T18:23:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:50 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:23:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:50 smithi099 ceph-mon[37557]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-10T18:23:50.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:50 smithi099 ceph-mon[37557]: Health check failed: Degraded data redundancy: 81/591 objects degraded (13.706%), 21 pgs degraded (PG_DEGRADED) 2024-09-10T18:23:51.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:51 smithi133 ceph-mon[31214]: osdmap e289: 8 total, 8 up, 8 in 2024-09-10T18:23:51.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:51 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:51.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:51 smithi133 ceph-mon[31214]: pgmap v395: 196 pgs: 1 unknown, 1 undersized+peered, 46 active+undersized, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 64 peering, 5 stale+active+clean, 7 undersized+peered+wait, 20 active+undersized+wait, 1 undersized+degraded+peered, 14 active+undersized+degraded, 31 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 81/591 objects degraded (13.706%) 2024-09-10T18:23:51.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:51 smithi099 ceph-mon[27037]: osdmap e289: 8 total, 8 up, 8 in 2024-09-10T18:23:51.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:51 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:51.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:51 smithi099 ceph-mon[27037]: pgmap v395: 196 pgs: 1 unknown, 1 undersized+peered, 46 active+undersized, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 64 peering, 5 stale+active+clean, 7 undersized+peered+wait, 20 active+undersized+wait, 1 undersized+degraded+peered, 14 active+undersized+degraded, 31 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 81/591 objects degraded (13.706%) 2024-09-10T18:23:51.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:51 smithi099 ceph-mon[37557]: osdmap e289: 8 total, 8 up, 8 in 2024-09-10T18:23:51.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:51 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:23:51.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:51 smithi099 ceph-mon[37557]: pgmap v395: 196 pgs: 1 unknown, 1 undersized+peered, 46 active+undersized, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 64 peering, 5 stale+active+clean, 7 undersized+peered+wait, 20 active+undersized+wait, 1 undersized+degraded+peered, 14 active+undersized+degraded, 31 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 81/591 objects degraded (13.706%) 2024-09-10T18:23:52.375 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_read_wait_for_complete PASSED [ 76%] 2024-09-10T18:23:52.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:52 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:52.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:52 smithi133 ceph-mon[31214]: osdmap e290: 8 total, 8 up, 8 in 2024-09-10T18:23:52.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:52 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:52.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:52 smithi099 ceph-mon[27037]: osdmap e290: 8 total, 8 up, 8 in 2024-09-10T18:23:52.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:52 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/451732401' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:23:52.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:52 smithi099 ceph-mon[37557]: osdmap e290: 8 total, 8 up, 8 in 2024-09-10T18:23:53.666 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:53 smithi099 ceph-mon[27037]: osdmap e291: 8 total, 8 up, 8 in 2024-09-10T18:23:53.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:53 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:23:53.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:53 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:53.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:53 smithi099 ceph-mon[27037]: pgmap v398: 164 pgs: 1 unknown, 39 active+undersized, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 53 peering, 5 stale+active+clean, 4 undersized+peered+wait, 17 active+undersized+wait, 1 undersized+degraded+peered, 14 active+undersized+degraded, 24 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 81/588 objects degraded (13.776%) 2024-09-10T18:23:53.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:53 smithi099 ceph-mon[37557]: osdmap e291: 8 total, 8 up, 8 in 2024-09-10T18:23:53.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:53 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:23:53.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:53 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:53.667 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:53 smithi099 ceph-mon[37557]: pgmap v398: 164 pgs: 1 unknown, 39 active+undersized, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 53 peering, 5 stale+active+clean, 4 undersized+peered+wait, 17 active+undersized+wait, 1 undersized+degraded+peered, 14 active+undersized+degraded, 24 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 81/588 objects degraded (13.776%) 2024-09-10T18:23:53.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:23:53 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:23:53.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:53 smithi133 ceph-mon[31214]: osdmap e291: 8 total, 8 up, 8 in 2024-09-10T18:23:53.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:53 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:23:53.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:53 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:23:53.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:53 smithi133 ceph-mon[31214]: pgmap v398: 164 pgs: 1 unknown, 39 active+undersized, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 53 peering, 5 stale+active+clean, 4 undersized+peered+wait, 17 active+undersized+wait, 1 undersized+degraded+peered, 14 active+undersized+degraded, 24 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 81/588 objects degraded (13.776%) 2024-09-10T18:23:54.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:23:53 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:23:53] "GET /metrics HTTP/1.1" 200 222867 "" "Prometheus/2.43.0" 2024-09-10T18:23:54.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:54 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:54.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:54 smithi133 ceph-mon[31214]: osdmap e292: 8 total, 8 up, 8 in 2024-09-10T18:23:54.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:54 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:54.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:54 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:54.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:54 smithi099 ceph-mon[27037]: osdmap e292: 8 total, 8 up, 8 in 2024-09-10T18:23:54.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:54 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:54.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:54 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:54.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:54 smithi099 ceph-mon[37557]: osdmap e292: 8 total, 8 up, 8 in 2024-09-10T18:23:54.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:54 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:23:55.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:55 smithi133 ceph-mon[31214]: osdmap e293: 8 total, 8 up, 8 in 2024-09-10T18:23:55.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:55 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-09-10T18:23:55.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:55 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:23:55.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:55 smithi133 ceph-mon[31214]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:23:55.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:55 smithi133 ceph-mon[31214]: pgmap v401: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-10T18:23:55.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:55 smithi099 ceph-mon[27037]: osdmap e293: 8 total, 8 up, 8 in 2024-09-10T18:23:55.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:55 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-09-10T18:23:55.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:55 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:23:55.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:55 smithi099 ceph-mon[27037]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:23:55.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:55 smithi099 ceph-mon[27037]: pgmap v401: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-10T18:23:55.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:55 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a[27033]: 2024-09-10T18:23:55.398+0000 7fe73a4f0640 -1 mon.a@0(leader).osd e294 definitely_dead 0 2024-09-10T18:23:55.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:55 smithi099 ceph-mon[37557]: osdmap e293: 8 total, 8 up, 8 in 2024-09-10T18:23:55.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:55 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-09-10T18:23:55.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:55 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:23:55.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:55 smithi099 ceph-mon[37557]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:23:55.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:55 smithi099 ceph-mon[37557]: pgmap v401: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-10T18:23:56.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:56 smithi133 ceph-mon[31214]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-10T18:23:56.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:56 smithi133 ceph-mon[31214]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 81/588 objects degraded (13.776%), 21 pgs degraded) 2024-09-10T18:23:56.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:56 smithi133 ceph-mon[31214]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:23:56.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:56 smithi133 ceph-mon[31214]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-10T18:23:56.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:56 smithi133 ceph-mon[31214]: osdmap e294: 8 total, 8 up, 8 in 2024-09-10T18:23:56.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:56 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-09-10T18:23:56.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:56 smithi133 ceph-mon[31214]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-09-10T18:23:56.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[27037]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-10T18:23:56.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[27037]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 81/588 objects degraded (13.776%), 21 pgs degraded) 2024-09-10T18:23:56.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[27037]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:23:56.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[27037]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-10T18:23:56.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[27037]: osdmap e294: 8 total, 8 up, 8 in 2024-09-10T18:23:56.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-09-10T18:23:56.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[27037]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-09-10T18:23:56.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[37557]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-10T18:23:56.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[37557]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 81/588 objects degraded (13.776%), 21 pgs degraded) 2024-09-10T18:23:56.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[37557]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:23:56.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[37557]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-10T18:23:56.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[37557]: osdmap e294: 8 total, 8 up, 8 in 2024-09-10T18:23:56.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-09-10T18:23:56.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:56 smithi099 ceph-mon[37557]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-09-10T18:23:57.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:23:57 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:23:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:23:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:57 smithi133 ceph-mon[31214]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-10T18:23:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:57 smithi133 ceph-mon[31214]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["5", "2", "4"]}]': finished 2024-09-10T18:23:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:57 smithi133 ceph-mon[31214]: osdmap e295: 8 total, 5 up, 8 in 2024-09-10T18:23:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:57 smithi133 ceph-mon[31214]: pgmap v404: 196 pgs: 56 stale+active+clean, 32 unknown, 108 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-10T18:23:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:57 smithi133 ceph-mon[31214]: osd.2 marked itself dead as of e295 2024-09-10T18:23:57.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:57 smithi133 ceph-mon[31214]: osdmap e296: 8 total, 5 up, 8 in 2024-09-10T18:23:57.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:57 smithi099 ceph-mon[27037]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-10T18:23:57.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:57 smithi099 ceph-mon[27037]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["5", "2", "4"]}]': finished 2024-09-10T18:23:57.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:57 smithi099 ceph-mon[27037]: osdmap e295: 8 total, 5 up, 8 in 2024-09-10T18:23:57.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:57 smithi099 ceph-mon[27037]: pgmap v404: 196 pgs: 56 stale+active+clean, 32 unknown, 108 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-10T18:23:57.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:57 smithi099 ceph-mon[27037]: osd.2 marked itself dead as of e295 2024-09-10T18:23:57.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:57 smithi099 ceph-mon[27037]: osdmap e296: 8 total, 5 up, 8 in 2024-09-10T18:23:57.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:57 smithi099 ceph-mon[37557]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-10T18:23:57.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:57 smithi099 ceph-mon[37557]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["5", "2", "4"]}]': finished 2024-09-10T18:23:57.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:57 smithi099 ceph-mon[37557]: osdmap e295: 8 total, 5 up, 8 in 2024-09-10T18:23:57.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:57 smithi099 ceph-mon[37557]: pgmap v404: 196 pgs: 56 stale+active+clean, 32 unknown, 108 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-10T18:23:57.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:57 smithi099 ceph-mon[37557]: osd.2 marked itself dead as of e295 2024-09-10T18:23:57.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:57 smithi099 ceph-mon[37557]: osdmap e296: 8 total, 5 up, 8 in 2024-09-10T18:23:58.255 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:23:57 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2[53449]: 2024-09-10T18:23:57.962+0000 7effdad09640 -1 osd.2 296 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:23:58.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:58 smithi133 ceph-mon[31214]: Monitor daemon marked osd.2 down, but it is still running 2024-09-10T18:23:58.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:58 smithi133 ceph-mon[31214]: map e295 wrongly marked me down at e295 2024-09-10T18:23:58.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:58 smithi133 ceph-mon[31214]: osd.4 marked itself dead as of e296 2024-09-10T18:23:58.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:58 smithi133 ceph-mon[31214]: osd.5 marked itself dead as of e296 2024-09-10T18:23:58.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:58 smithi099 ceph-mon[27037]: Monitor daemon marked osd.2 down, but it is still running 2024-09-10T18:23:58.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:58 smithi099 ceph-mon[27037]: map e295 wrongly marked me down at e295 2024-09-10T18:23:58.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:58 smithi099 ceph-mon[27037]: osd.4 marked itself dead as of e296 2024-09-10T18:23:58.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:58 smithi099 ceph-mon[27037]: osd.5 marked itself dead as of e296 2024-09-10T18:23:58.757 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:23:58 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2[53449]: 2024-09-10T18:23:58.431+0000 7effcc2fe640 -1 osd.2 297 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:23:58.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:58 smithi099 ceph-mon[37557]: Monitor daemon marked osd.2 down, but it is still running 2024-09-10T18:23:58.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:58 smithi099 ceph-mon[37557]: map e295 wrongly marked me down at e295 2024-09-10T18:23:58.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:58 smithi099 ceph-mon[37557]: osd.4 marked itself dead as of e296 2024-09-10T18:23:58.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:58 smithi099 ceph-mon[37557]: osd.5 marked itself dead as of e296 2024-09-10T18:23:59.090 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:23:58 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:23:58.837+0000 7fe09667c640 -1 osd.4 297 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:23:59.428 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:23:59 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:23:59.088+0000 7fca32ffd640 -1 osd.5 297 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:23:59.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:59 smithi133 ceph-mon[31214]: Monitor daemon marked osd.4 down, but it is still running 2024-09-10T18:23:59.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:59 smithi133 ceph-mon[31214]: map e296 wrongly marked me down at e295 2024-09-10T18:23:59.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:59 smithi133 ceph-mon[31214]: Monitor daemon marked osd.5 down, but it is still running 2024-09-10T18:23:59.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:59 smithi133 ceph-mon[31214]: map e296 wrongly marked me down at e295 2024-09-10T18:23:59.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:59 smithi133 ceph-mon[31214]: pgmap v406: 196 pgs: 23 active+undersized, 8 undersized+degraded+peered+wait, 23 active+undersized+degraded+wait, 1 stale+active+clean, 36 undersized+peered+wait, 59 active+undersized+wait, 7 active+undersized+degraded, 39 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 254 B/s wr, 0 op/s; 176/591 objects degraded (29.780%) 2024-09-10T18:23:59.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:59 smithi133 ceph-mon[31214]: Health check failed: Reduced data availability: 4 pgs inactive (PG_AVAILABILITY) 2024-09-10T18:23:59.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:59 smithi133 ceph-mon[31214]: Health check failed: Degraded data redundancy: 176/591 objects degraded (29.780%), 38 pgs degraded (PG_DEGRADED) 2024-09-10T18:23:59.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:59 smithi133 ceph-mon[31214]: osdmap e297: 8 total, 5 up, 8 in 2024-09-10T18:23:59.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:23:59 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:59.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[27037]: Monitor daemon marked osd.4 down, but it is still running 2024-09-10T18:23:59.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[27037]: map e296 wrongly marked me down at e295 2024-09-10T18:23:59.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[27037]: Monitor daemon marked osd.5 down, but it is still running 2024-09-10T18:23:59.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[27037]: map e296 wrongly marked me down at e295 2024-09-10T18:23:59.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[27037]: pgmap v406: 196 pgs: 23 active+undersized, 8 undersized+degraded+peered+wait, 23 active+undersized+degraded+wait, 1 stale+active+clean, 36 undersized+peered+wait, 59 active+undersized+wait, 7 active+undersized+degraded, 39 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 254 B/s wr, 0 op/s; 176/591 objects degraded (29.780%) 2024-09-10T18:23:59.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[27037]: Health check failed: Reduced data availability: 4 pgs inactive (PG_AVAILABILITY) 2024-09-10T18:23:59.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[27037]: Health check failed: Degraded data redundancy: 176/591 objects degraded (29.780%), 38 pgs degraded (PG_DEGRADED) 2024-09-10T18:23:59.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[27037]: osdmap e297: 8 total, 5 up, 8 in 2024-09-10T18:23:59.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:23:59.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[37557]: Monitor daemon marked osd.4 down, but it is still running 2024-09-10T18:23:59.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[37557]: map e296 wrongly marked me down at e295 2024-09-10T18:23:59.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[37557]: Monitor daemon marked osd.5 down, but it is still running 2024-09-10T18:23:59.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[37557]: map e296 wrongly marked me down at e295 2024-09-10T18:23:59.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[37557]: pgmap v406: 196 pgs: 23 active+undersized, 8 undersized+degraded+peered+wait, 23 active+undersized+degraded+wait, 1 stale+active+clean, 36 undersized+peered+wait, 59 active+undersized+wait, 7 active+undersized+degraded, 39 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 254 B/s wr, 0 op/s; 176/591 objects degraded (29.780%) 2024-09-10T18:23:59.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[37557]: Health check failed: Reduced data availability: 4 pgs inactive (PG_AVAILABILITY) 2024-09-10T18:23:59.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[37557]: Health check failed: Degraded data redundancy: 176/591 objects degraded (29.780%), 38 pgs degraded (PG_DEGRADED) 2024-09-10T18:23:59.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[37557]: osdmap e297: 8 total, 5 up, 8 in 2024-09-10T18:23:59.757 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:23:59 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:00.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:00 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:00.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:00 smithi133 ceph-mon[31214]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:00.741 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:24:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:24:00.440+0000 7fca2be0e640 -1 osd.5 298 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:24:00.741 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:24:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:24:00.440+0000 7fe08f4af640 -1 osd.4 298 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:24:00.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:00 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:00.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:00 smithi099 ceph-mon[27037]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:00.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:00 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:00.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:00 smithi099 ceph-mon[37557]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:00.755 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:24:00 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2[53449]: 2024-09-10T18:24:00.440+0000 7effd4b1c640 -1 osd.2 298 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:24:01.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:01 smithi133 ceph-mon[31214]: pgmap v408: 196 pgs: 23 active+undersized, 8 undersized+degraded+peered+wait, 23 active+undersized+degraded+wait, 5 stale+active+clean, 36 undersized+peered+wait, 59 active+undersized+wait, 7 active+undersized+degraded, 35 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 204 B/s wr, 0 op/s; 176/591 objects degraded (29.780%) 2024-09-10T18:24:01.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:01 smithi133 ceph-mon[31214]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-10T18:24:01.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:01 smithi133 ceph-mon[31214]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:01.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:01 smithi133 ceph-mon[31214]: osdmap e298: 8 total, 5 up, 8 in 2024-09-10T18:24:01.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:01 smithi099 ceph-mon[27037]: pgmap v408: 196 pgs: 23 active+undersized, 8 undersized+degraded+peered+wait, 23 active+undersized+degraded+wait, 5 stale+active+clean, 36 undersized+peered+wait, 59 active+undersized+wait, 7 active+undersized+degraded, 35 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 204 B/s wr, 0 op/s; 176/591 objects degraded (29.780%) 2024-09-10T18:24:01.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:01 smithi099 ceph-mon[27037]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-10T18:24:01.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:01 smithi099 ceph-mon[27037]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:01.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:01 smithi099 ceph-mon[27037]: osdmap e298: 8 total, 5 up, 8 in 2024-09-10T18:24:01.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:01 smithi099 ceph-mon[37557]: pgmap v408: 196 pgs: 23 active+undersized, 8 undersized+degraded+peered+wait, 23 active+undersized+degraded+wait, 5 stale+active+clean, 36 undersized+peered+wait, 59 active+undersized+wait, 7 active+undersized+degraded, 35 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 204 B/s wr, 0 op/s; 176/591 objects degraded (29.780%) 2024-09-10T18:24:01.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:01 smithi099 ceph-mon[37557]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-10T18:24:01.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:01 smithi099 ceph-mon[37557]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:01.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:01 smithi099 ceph-mon[37557]: osdmap e298: 8 total, 5 up, 8 in 2024-09-10T18:24:02.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:02 smithi133 ceph-mon[31214]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-10T18:24:02.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:02 smithi133 ceph-mon[31214]: osd.4 v1:172.21.15.133:6800/81144752 boot 2024-09-10T18:24:02.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:02 smithi133 ceph-mon[31214]: osd.5 v1:172.21.15.133:6804/2255028820 boot 2024-09-10T18:24:02.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:02 smithi133 ceph-mon[31214]: osd.2 v1:172.21.15.99:6809/592364138 boot 2024-09-10T18:24:02.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:02 smithi133 ceph-mon[31214]: osdmap e299: 8 total, 8 up, 8 in 2024-09-10T18:24:02.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:02 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:24:02.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:02 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:24:02.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:02 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:24:02.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[27037]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-10T18:24:02.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[27037]: osd.4 v1:172.21.15.133:6800/81144752 boot 2024-09-10T18:24:02.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[27037]: osd.5 v1:172.21.15.133:6804/2255028820 boot 2024-09-10T18:24:02.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[27037]: osd.2 v1:172.21.15.99:6809/592364138 boot 2024-09-10T18:24:02.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[27037]: osdmap e299: 8 total, 8 up, 8 in 2024-09-10T18:24:02.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:24:02.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:24:02.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:24:02.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[37557]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-10T18:24:02.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[37557]: osd.4 v1:172.21.15.133:6800/81144752 boot 2024-09-10T18:24:02.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[37557]: osd.5 v1:172.21.15.133:6804/2255028820 boot 2024-09-10T18:24:02.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[37557]: osd.2 v1:172.21.15.99:6809/592364138 boot 2024-09-10T18:24:02.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[37557]: osdmap e299: 8 total, 8 up, 8 in 2024-09-10T18:24:02.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:24:02.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:24:02.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:02 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:24:03.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:24:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:24:03.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:03 smithi133 ceph-mon[31214]: pgmap v411: 196 pgs: 23 active+undersized, 8 undersized+degraded+peered+wait, 23 active+undersized+degraded+wait, 5 stale+active+clean, 36 undersized+peered+wait, 59 active+undersized+wait, 7 active+undersized+degraded, 35 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 204 B/s wr, 0 op/s; 176/591 objects degraded (29.780%) 2024-09-10T18:24:03.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:03 smithi133 ceph-mon[31214]: osdmap e300: 8 total, 8 up, 8 in 2024-09-10T18:24:03.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:03 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:03.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:03 smithi133 ceph-mon[31214]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:03.755 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:24:03 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:24:03] "GET /metrics HTTP/1.1" 200 222867 "" "Prometheus/2.43.0" 2024-09-10T18:24:03.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:03 smithi099 ceph-mon[27037]: pgmap v411: 196 pgs: 23 active+undersized, 8 undersized+degraded+peered+wait, 23 active+undersized+degraded+wait, 5 stale+active+clean, 36 undersized+peered+wait, 59 active+undersized+wait, 7 active+undersized+degraded, 35 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 204 B/s wr, 0 op/s; 176/591 objects degraded (29.780%) 2024-09-10T18:24:03.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:03 smithi099 ceph-mon[27037]: osdmap e300: 8 total, 8 up, 8 in 2024-09-10T18:24:03.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:03 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:03.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:03 smithi099 ceph-mon[27037]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:03.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:03 smithi099 ceph-mon[37557]: pgmap v411: 196 pgs: 23 active+undersized, 8 undersized+degraded+peered+wait, 23 active+undersized+degraded+wait, 5 stale+active+clean, 36 undersized+peered+wait, 59 active+undersized+wait, 7 active+undersized+degraded, 35 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 204 B/s wr, 0 op/s; 176/591 objects degraded (29.780%) 2024-09-10T18:24:03.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:03 smithi099 ceph-mon[37557]: osdmap e300: 8 total, 8 up, 8 in 2024-09-10T18:24:03.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:03 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1389936270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:03.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:03 smithi099 ceph-mon[37557]: from='client.35078 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:04.465 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_read_wait_for_complete_and_cb PASSED [ 77%] 2024-09-10T18:24:04.740 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:04 smithi133 ceph-mon[31214]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:04.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:04 smithi133 ceph-mon[31214]: osdmap e301: 8 total, 8 up, 8 in 2024-09-10T18:24:04.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:04 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:04.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:04 smithi099 ceph-mon[27037]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:04.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:04 smithi099 ceph-mon[27037]: osdmap e301: 8 total, 8 up, 8 in 2024-09-10T18:24:04.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:04 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:04.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:04 smithi099 ceph-mon[37557]: from='client.35078 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:04.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:04 smithi099 ceph-mon[37557]: osdmap e301: 8 total, 8 up, 8 in 2024-09-10T18:24:04.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:04 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:05.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:05 smithi133 ceph-mon[31214]: pgmap v414: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 2 op/s; 54 B/s, 2 objects/s recovering 2024-09-10T18:24:05.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:05 smithi133 ceph-mon[31214]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive) 2024-09-10T18:24:05.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:05 smithi133 ceph-mon[31214]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 176/591 objects degraded (29.780%), 38 pgs degraded) 2024-09-10T18:24:05.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:05 smithi133 ceph-mon[31214]: osdmap e302: 8 total, 8 up, 8 in 2024-09-10T18:24:05.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:05 smithi099 ceph-mon[27037]: pgmap v414: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 2 op/s; 54 B/s, 2 objects/s recovering 2024-09-10T18:24:05.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:05 smithi099 ceph-mon[27037]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive) 2024-09-10T18:24:05.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:05 smithi099 ceph-mon[27037]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 176/591 objects degraded (29.780%), 38 pgs degraded) 2024-09-10T18:24:05.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:05 smithi099 ceph-mon[27037]: osdmap e302: 8 total, 8 up, 8 in 2024-09-10T18:24:05.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:05 smithi099 ceph-mon[37557]: pgmap v414: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 2 op/s; 54 B/s, 2 objects/s recovering 2024-09-10T18:24:05.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:05 smithi099 ceph-mon[37557]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive) 2024-09-10T18:24:05.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:05 smithi099 ceph-mon[37557]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 176/591 objects degraded (29.780%), 38 pgs degraded) 2024-09-10T18:24:05.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:05 smithi099 ceph-mon[37557]: osdmap e302: 8 total, 8 up, 8 in 2024-09-10T18:24:06.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:06 smithi133 ceph-mon[31214]: osdmap e303: 8 total, 8 up, 8 in 2024-09-10T18:24:06.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:06 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "bar", "format": "json"}]: dispatch 2024-09-10T18:24:06.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:06 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:24:06.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:06 smithi133 ceph-mon[31214]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:24:06.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:06 smithi099 ceph-mon[37557]: osdmap e303: 8 total, 8 up, 8 in 2024-09-10T18:24:06.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:06 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "bar", "format": "json"}]: dispatch 2024-09-10T18:24:06.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:06 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:24:06.755 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:06 smithi099 ceph-mon[37557]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:24:06.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:06 smithi099 ceph-mon[27037]: osdmap e303: 8 total, 8 up, 8 in 2024-09-10T18:24:06.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:06 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "bar", "format": "json"}]: dispatch 2024-09-10T18:24:06.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:06 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:24:06.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:06 smithi099 ceph-mon[27037]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-10T18:24:06.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:06 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a[27033]: 2024-09-10T18:24:06.474+0000 7fe73a4f0640 -1 mon.a@0(leader).osd e304 definitely_dead 0 2024-09-10T18:24:07.477 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:24:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:24:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:24:07.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:07 smithi133 ceph-mon[31214]: pgmap v417: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s; 54 B/s, 2 objects/s recovering 2024-09-10T18:24:07.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:07 smithi133 ceph-mon[31214]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:24:07.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:07 smithi133 ceph-mon[31214]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-10T18:24:07.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:07 smithi133 ceph-mon[31214]: osdmap e304: 8 total, 8 up, 8 in 2024-09-10T18:24:07.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:07 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["1", "6", "4"]}]: dispatch 2024-09-10T18:24:07.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:07 smithi133 ceph-mon[31214]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["1", "6", "4"]}]: dispatch 2024-09-10T18:24:07.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:07 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:07.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:07 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:07.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[27037]: pgmap v417: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s; 54 B/s, 2 objects/s recovering 2024-09-10T18:24:07.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[27037]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:24:07.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[27037]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-10T18:24:07.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[27037]: osdmap e304: 8 total, 8 up, 8 in 2024-09-10T18:24:07.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["1", "6", "4"]}]: dispatch 2024-09-10T18:24:07.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[27037]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["1", "6", "4"]}]: dispatch 2024-09-10T18:24:07.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:07.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:07.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[37557]: pgmap v417: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s; 54 B/s, 2 objects/s recovering 2024-09-10T18:24:07.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[37557]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-10T18:24:07.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[37557]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-10T18:24:07.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[37557]: osdmap e304: 8 total, 8 up, 8 in 2024-09-10T18:24:07.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["1", "6", "4"]}]: dispatch 2024-09-10T18:24:07.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[37557]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["1", "6", "4"]}]: dispatch 2024-09-10T18:24:07.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:07.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:07 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:08.241 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:24:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:24:07.840+0000 7fe097ea1640 -1 osd.4 305 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:24:08.741 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:24:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:24:08.476+0000 7fe086c91640 -1 osd.4 306 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:24:08.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:08 smithi133 ceph-mon[31214]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-10T18:24:08.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:08 smithi133 ceph-mon[31214]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["1", "6", "4"]}]': finished 2024-09-10T18:24:08.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:08 smithi133 ceph-mon[31214]: osdmap e305: 8 total, 5 up, 8 in 2024-09-10T18:24:08.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:08 smithi133 ceph-mon[31214]: Monitor daemon marked osd.4 down, but it is still running 2024-09-10T18:24:08.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:08 smithi133 ceph-mon[31214]: map e305 wrongly marked me down at e305 2024-09-10T18:24:08.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:08 smithi133 ceph-mon[31214]: osd.4 marked itself dead as of e305 2024-09-10T18:24:08.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:08 smithi099 ceph-mon[27037]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-10T18:24:08.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:08 smithi099 ceph-mon[27037]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["1", "6", "4"]}]': finished 2024-09-10T18:24:08.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:08 smithi099 ceph-mon[27037]: osdmap e305: 8 total, 5 up, 8 in 2024-09-10T18:24:08.755 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:08 smithi099 ceph-mon[27037]: Monitor daemon marked osd.4 down, but it is still running 2024-09-10T18:24:08.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:08 smithi099 ceph-mon[27037]: map e305 wrongly marked me down at e305 2024-09-10T18:24:08.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:08 smithi099 ceph-mon[27037]: osd.4 marked itself dead as of e305 2024-09-10T18:24:08.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:08 smithi099 ceph-mon[37557]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-10T18:24:08.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:08 smithi099 ceph-mon[37557]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["1", "6", "4"]}]': finished 2024-09-10T18:24:08.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:08 smithi099 ceph-mon[37557]: osdmap e305: 8 total, 5 up, 8 in 2024-09-10T18:24:08.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:08 smithi099 ceph-mon[37557]: Monitor daemon marked osd.4 down, but it is still running 2024-09-10T18:24:08.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:08 smithi099 ceph-mon[37557]: map e305 wrongly marked me down at e305 2024-09-10T18:24:08.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:08 smithi099 ceph-mon[37557]: osd.4 marked itself dead as of e305 2024-09-10T18:24:09.657 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1[47455]: 2024-09-10T18:24:09.352+0000 7ff438577640 -1 osd.1 306 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:24:09.657 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1[47455]: 2024-09-10T18:24:09.501+0000 7ff42b391640 -1 osd.1 307 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:24:09.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-mon[27037]: pgmap v420: 196 pgs: 130 peering, 2 stale+active+clean, 64 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:24:09.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-mon[27037]: osdmap e306: 8 total, 5 up, 8 in 2024-09-10T18:24:09.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-mon[27037]: Health check failed: Reduced data availability: 2 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2024-09-10T18:24:09.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-mon[27037]: Monitor daemon marked osd.1 down, but it is still running 2024-09-10T18:24:09.658 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-mon[27037]: map e306 wrongly marked me down at e305 2024-09-10T18:24:09.658 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-mon[27037]: osd.1 marked itself dead as of e306 2024-09-10T18:24:09.658 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-mon[37557]: pgmap v420: 196 pgs: 130 peering, 2 stale+active+clean, 64 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:24:09.658 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-mon[37557]: osdmap e306: 8 total, 5 up, 8 in 2024-09-10T18:24:09.658 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-mon[37557]: Health check failed: Reduced data availability: 2 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2024-09-10T18:24:09.658 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-mon[37557]: Monitor daemon marked osd.1 down, but it is still running 2024-09-10T18:24:09.658 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-mon[37557]: map e306 wrongly marked me down at e305 2024-09-10T18:24:09.658 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:09 smithi099 ceph-mon[37557]: osd.1 marked itself dead as of e306 2024-09-10T18:24:09.990 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:24:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:24:09.503+0000 7fe086c91640 -1 osd.4 307 osdmap NOUP flag is set, waiting for it to clear 2024-09-10T18:24:09.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:09 smithi133 ceph-mon[31214]: pgmap v420: 196 pgs: 130 peering, 2 stale+active+clean, 64 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:24:09.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:09 smithi133 ceph-mon[31214]: osdmap e306: 8 total, 5 up, 8 in 2024-09-10T18:24:09.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:09 smithi133 ceph-mon[31214]: Health check failed: Reduced data availability: 2 pgs inactive, 6 pgs peering (PG_AVAILABILITY) 2024-09-10T18:24:09.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:09 smithi133 ceph-mon[31214]: Monitor daemon marked osd.1 down, but it is still running 2024-09-10T18:24:09.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:09 smithi133 ceph-mon[31214]: map e306 wrongly marked me down at e305 2024-09-10T18:24:09.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:09 smithi133 ceph-mon[31214]: osd.1 marked itself dead as of e306 2024-09-10T18:24:10.755 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:24:10 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1[47455]: 2024-09-10T18:24:10.509+0000 7ff433baf640 -1 osd.1 308 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:24:10.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:10 smithi099 ceph-mon[27037]: osdmap e307: 8 total, 5 up, 8 in 2024-09-10T18:24:10.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:10 smithi099 ceph-mon[27037]: osd.6 marked itself dead as of e307 2024-09-10T18:24:10.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:10 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:10.756 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:10 smithi099 ceph-mon[27037]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:10.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:10 smithi099 ceph-mon[37557]: osdmap e307: 8 total, 5 up, 8 in 2024-09-10T18:24:10.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:10 smithi099 ceph-mon[37557]: osd.6 marked itself dead as of e307 2024-09-10T18:24:10.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:10 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:10.756 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:10 smithi099 ceph-mon[37557]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:10.991 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:24:10 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:24:10.567+0000 7f48db2c1640 -1 osd.6 308 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:24:10.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:10 smithi133 ceph-mon[31214]: osdmap e307: 8 total, 5 up, 8 in 2024-09-10T18:24:10.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:10 smithi133 ceph-mon[31214]: osd.6 marked itself dead as of e307 2024-09-10T18:24:10.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:10 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:10.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:10 smithi133 ceph-mon[31214]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:10.991 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:24:10 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:24:10.508+0000 7fe08f4af640 -1 osd.4 308 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:24:11.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:11 smithi133 ceph-mon[31214]: pgmap v423: 196 pgs: 130 peering, 24 stale+active+clean, 42 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:24:11.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:11 smithi133 ceph-mon[31214]: Monitor daemon marked osd.6 down, but it is still running 2024-09-10T18:24:11.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:11 smithi133 ceph-mon[31214]: map e307 wrongly marked me down at e305 2024-09-10T18:24:11.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:11 smithi133 ceph-mon[31214]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-10T18:24:11.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:11 smithi133 ceph-mon[31214]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:11.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:11 smithi133 ceph-mon[31214]: osdmap e308: 8 total, 5 up, 8 in 2024-09-10T18:24:12.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:11 smithi099 ceph-mon[27037]: pgmap v423: 196 pgs: 130 peering, 24 stale+active+clean, 42 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:24:12.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:11 smithi099 ceph-mon[27037]: Monitor daemon marked osd.6 down, but it is still running 2024-09-10T18:24:12.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:11 smithi099 ceph-mon[27037]: map e307 wrongly marked me down at e305 2024-09-10T18:24:12.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:11 smithi099 ceph-mon[27037]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-10T18:24:12.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:11 smithi099 ceph-mon[27037]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:12.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:11 smithi099 ceph-mon[27037]: osdmap e308: 8 total, 5 up, 8 in 2024-09-10T18:24:12.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:11 smithi099 ceph-mon[37557]: pgmap v423: 196 pgs: 130 peering, 24 stale+active+clean, 42 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:24:12.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:11 smithi099 ceph-mon[37557]: Monitor daemon marked osd.6 down, but it is still running 2024-09-10T18:24:12.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:11 smithi099 ceph-mon[37557]: map e307 wrongly marked me down at e305 2024-09-10T18:24:12.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:11 smithi099 ceph-mon[37557]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-10T18:24:12.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:11 smithi099 ceph-mon[37557]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:12.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:11 smithi099 ceph-mon[37557]: osdmap e308: 8 total, 5 up, 8 in 2024-09-10T18:24:12.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:12 smithi133 ceph-mon[31214]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-10T18:24:12.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:12 smithi133 ceph-mon[31214]: osd.4 v1:172.21.15.133:6800/81144752 boot 2024-09-10T18:24:12.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:12 smithi133 ceph-mon[31214]: osd.1 v1:172.21.15.99:6805/719640887 boot 2024-09-10T18:24:12.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:12 smithi133 ceph-mon[31214]: osd.6 v1:172.21.15.133:6808/2733605656 boot 2024-09-10T18:24:12.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:12 smithi133 ceph-mon[31214]: osdmap e309: 8 total, 8 up, 8 in 2024-09-10T18:24:12.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:12 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:24:12.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:12 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:24:12.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:12 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:24:12.992 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:12 smithi133 ceph-mon[31214]: osdmap e310: 8 total, 8 up, 8 in 2024-09-10T18:24:13.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[27037]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-10T18:24:13.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[27037]: osd.4 v1:172.21.15.133:6800/81144752 boot 2024-09-10T18:24:13.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[27037]: osd.1 v1:172.21.15.99:6805/719640887 boot 2024-09-10T18:24:13.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[27037]: osd.6 v1:172.21.15.133:6808/2733605656 boot 2024-09-10T18:24:13.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[27037]: osdmap e309: 8 total, 8 up, 8 in 2024-09-10T18:24:13.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:24:13.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:24:13.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:24:13.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[27037]: osdmap e310: 8 total, 8 up, 8 in 2024-09-10T18:24:13.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[37557]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-10T18:24:13.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[37557]: osd.4 v1:172.21.15.133:6800/81144752 boot 2024-09-10T18:24:13.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[37557]: osd.1 v1:172.21.15.99:6805/719640887 boot 2024-09-10T18:24:13.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[37557]: osd.6 v1:172.21.15.133:6808/2733605656 boot 2024-09-10T18:24:13.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[37557]: osdmap e309: 8 total, 8 up, 8 in 2024-09-10T18:24:13.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:24:13.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:24:13.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:24:13.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:12 smithi099 ceph-mon[37557]: osdmap e310: 8 total, 8 up, 8 in 2024-09-10T18:24:13.892 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:13 smithi133 ceph-mon[31214]: pgmap v426: 196 pgs: 130 peering, 24 stale+active+clean, 42 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:13.892 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:24:13 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:24:14.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:13 smithi099 ceph-mon[27037]: pgmap v426: 196 pgs: 130 peering, 24 stale+active+clean, 42 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:14.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:24:13 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:24:13] "GET /metrics HTTP/1.1" 200 225292 "" "Prometheus/2.43.0" 2024-09-10T18:24:14.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:13 smithi099 ceph-mon[37557]: pgmap v426: 196 pgs: 130 peering, 24 stale+active+clean, 42 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:14.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:14 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:14.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:14 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:14.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:14 smithi133 ceph-mon[31214]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:15.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:14 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:15.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:14 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:15.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:14 smithi099 ceph-mon[27037]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:15.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:14 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:15.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:14 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3771986787' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:15.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:14 smithi099 ceph-mon[37557]: from='client.35084 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:15.556 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_read_wait_for_complete_and_cb_error PASSED [ 78%] 2024-09-10T18:24:15.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:15 smithi133 ceph-mon[31214]: pgmap v428: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 B/s, 2 objects/s recovering 2024-09-10T18:24:15.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:15 smithi133 ceph-mon[31214]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 6 pgs peering) 2024-09-10T18:24:15.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:15 smithi133 ceph-mon[31214]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:15.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:15 smithi133 ceph-mon[31214]: osdmap e311: 8 total, 8 up, 8 in 2024-09-10T18:24:16.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:15 smithi099 ceph-mon[27037]: pgmap v428: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 B/s, 2 objects/s recovering 2024-09-10T18:24:16.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:15 smithi099 ceph-mon[27037]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 6 pgs peering) 2024-09-10T18:24:16.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:15 smithi099 ceph-mon[27037]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:16.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:15 smithi099 ceph-mon[27037]: osdmap e311: 8 total, 8 up, 8 in 2024-09-10T18:24:16.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:15 smithi099 ceph-mon[37557]: pgmap v428: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 B/s, 2 objects/s recovering 2024-09-10T18:24:16.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:15 smithi099 ceph-mon[37557]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 6 pgs peering) 2024-09-10T18:24:16.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:15 smithi099 ceph-mon[37557]: from='client.35084 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:16.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:15 smithi099 ceph-mon[37557]: osdmap e311: 8 total, 8 up, 8 in 2024-09-10T18:24:16.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:16 smithi133 ceph-mon[31214]: osdmap e312: 8 total, 8 up, 8 in 2024-09-10T18:24:17.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:16 smithi099 ceph-mon[27037]: osdmap e312: 8 total, 8 up, 8 in 2024-09-10T18:24:17.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:16 smithi099 ceph-mon[37557]: osdmap e312: 8 total, 8 up, 8 in 2024-09-10T18:24:17.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:24:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:24:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:24:17.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:17 smithi133 ceph-mon[31214]: pgmap v431: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 B/s, 2 objects/s recovering 2024-09-10T18:24:17.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:17 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:17.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:17 smithi133 ceph-mon[31214]: osdmap e313: 8 total, 8 up, 8 in 2024-09-10T18:24:18.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:17 smithi099 ceph-mon[27037]: pgmap v431: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 B/s, 2 objects/s recovering 2024-09-10T18:24:18.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:17 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:18.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:17 smithi099 ceph-mon[27037]: osdmap e313: 8 total, 8 up, 8 in 2024-09-10T18:24:18.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:17 smithi099 ceph-mon[37557]: pgmap v431: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 B/s, 2 objects/s recovering 2024-09-10T18:24:18.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:17 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:18.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:17 smithi099 ceph-mon[37557]: osdmap e313: 8 total, 8 up, 8 in 2024-09-10T18:24:18.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:18 smithi133 ceph-mon[31214]: osdmap e314: 8 total, 8 up, 8 in 2024-09-10T18:24:18.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:18 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2350029164' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:19.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:18 smithi099 ceph-mon[27037]: osdmap e314: 8 total, 8 up, 8 in 2024-09-10T18:24:19.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:18 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2350029164' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:19.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:18 smithi099 ceph-mon[37557]: osdmap e314: 8 total, 8 up, 8 in 2024-09-10T18:24:19.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:18 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2350029164' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:19.579 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_lock PASSED [ 79%] 2024-09-10T18:24:19.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:19 smithi133 ceph-mon[31214]: pgmap v434: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-10T18:24:19.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:19 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2350029164' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:19.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:19 smithi133 ceph-mon[31214]: osdmap e315: 8 total, 8 up, 8 in 2024-09-10T18:24:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:19 smithi099 ceph-mon[27037]: pgmap v434: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-10T18:24:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:19 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2350029164' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:19 smithi099 ceph-mon[27037]: osdmap e315: 8 total, 8 up, 8 in 2024-09-10T18:24:20.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:19 smithi099 ceph-mon[37557]: pgmap v434: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-10T18:24:20.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:19 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2350029164' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:20.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:19 smithi099 ceph-mon[37557]: osdmap e315: 8 total, 8 up, 8 in 2024-09-10T18:24:20.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:20 smithi133 ceph-mon[31214]: osdmap e316: 8 total, 8 up, 8 in 2024-09-10T18:24:21.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:20 smithi099 ceph-mon[27037]: osdmap e316: 8 total, 8 up, 8 in 2024-09-10T18:24:21.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:20 smithi099 ceph-mon[37557]: osdmap e316: 8 total, 8 up, 8 in 2024-09-10T18:24:21.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:21 smithi133 ceph-mon[31214]: pgmap v437: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:21.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:21 smithi133 ceph-mon[31214]: osdmap e317: 8 total, 8 up, 8 in 2024-09-10T18:24:22.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:21 smithi099 ceph-mon[27037]: pgmap v437: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:22.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:21 smithi099 ceph-mon[27037]: osdmap e317: 8 total, 8 up, 8 in 2024-09-10T18:24:22.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:21 smithi099 ceph-mon[37557]: pgmap v437: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:22.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:21 smithi099 ceph-mon[37557]: osdmap e317: 8 total, 8 up, 8 in 2024-09-10T18:24:22.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:22 smithi133 ceph-mon[31214]: osdmap e318: 8 total, 8 up, 8 in 2024-09-10T18:24:22.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:22 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2488512694' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:22.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:22 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:22.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:22 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:23.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:22 smithi099 ceph-mon[27037]: osdmap e318: 8 total, 8 up, 8 in 2024-09-10T18:24:23.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:22 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2488512694' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:23.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:22 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:23.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:22 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:23.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:22 smithi099 ceph-mon[37557]: osdmap e318: 8 total, 8 up, 8 in 2024-09-10T18:24:23.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:22 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2488512694' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:23.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:22 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:23.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:22 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:23.612 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_execute PASSED [ 80%] 2024-09-10T18:24:23.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:24:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:24:23.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:23 smithi133 ceph-mon[31214]: pgmap v440: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:23.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:23 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2488512694' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:23.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:23 smithi133 ceph-mon[31214]: osdmap e319: 8 total, 8 up, 8 in 2024-09-10T18:24:23.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:23 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:23.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:23 smithi133 ceph-mon[31214]: osdmap e320: 8 total, 8 up, 8 in 2024-09-10T18:24:24.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:24:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:24:23] "GET /metrics HTTP/1.1" 200 225302 "" "Prometheus/2.43.0" 2024-09-10T18:24:24.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:23 smithi099 ceph-mon[27037]: pgmap v440: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:24.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:23 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2488512694' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:24.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:23 smithi099 ceph-mon[27037]: osdmap e319: 8 total, 8 up, 8 in 2024-09-10T18:24:24.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:23 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:24.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:23 smithi099 ceph-mon[27037]: osdmap e320: 8 total, 8 up, 8 in 2024-09-10T18:24:24.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:23 smithi099 ceph-mon[37557]: pgmap v440: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:24.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:23 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2488512694' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:24.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:23 smithi099 ceph-mon[37557]: osdmap e319: 8 total, 8 up, 8 in 2024-09-10T18:24:24.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:23 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:24.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:23 smithi099 ceph-mon[37557]: osdmap e320: 8 total, 8 up, 8 in 2024-09-10T18:24:24.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:24 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:25.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:24 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:25.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:24 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:25.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:25 smithi133 ceph-mon[31214]: pgmap v443: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:25.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:25 smithi133 ceph-mon[31214]: osdmap e321: 8 total, 8 up, 8 in 2024-09-10T18:24:26.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:25 smithi099 ceph-mon[27037]: pgmap v443: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:26.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:25 smithi099 ceph-mon[27037]: osdmap e321: 8 total, 8 up, 8 in 2024-09-10T18:24:26.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:25 smithi099 ceph-mon[37557]: pgmap v443: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:26.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:25 smithi099 ceph-mon[37557]: osdmap e321: 8 total, 8 up, 8 in 2024-09-10T18:24:26.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:26 smithi133 ceph-mon[31214]: osdmap e322: 8 total, 8 up, 8 in 2024-09-10T18:24:26.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:26 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3392885592' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:27.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:26 smithi099 ceph-mon[27037]: osdmap e322: 8 total, 8 up, 8 in 2024-09-10T18:24:27.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:26 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3392885592' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:27.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:26 smithi099 ceph-mon[37557]: osdmap e322: 8 total, 8 up, 8 in 2024-09-10T18:24:27.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:26 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3392885592' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:27.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:24:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:24:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:24:27.651 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_execute PASSED [ 82%] 2024-09-10T18:24:27.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:27 smithi133 ceph-mon[31214]: pgmap v446: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:27.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:27 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3392885592' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:27.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:27 smithi133 ceph-mon[31214]: osdmap e323: 8 total, 8 up, 8 in 2024-09-10T18:24:28.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:27 smithi099 ceph-mon[27037]: pgmap v446: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:28.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:27 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3392885592' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:28.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:27 smithi099 ceph-mon[27037]: osdmap e323: 8 total, 8 up, 8 in 2024-09-10T18:24:28.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:27 smithi099 ceph-mon[37557]: pgmap v446: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:28.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:27 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3392885592' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:28.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:27 smithi099 ceph-mon[37557]: osdmap e323: 8 total, 8 up, 8 in 2024-09-10T18:24:28.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:28 smithi133 ceph-mon[31214]: osdmap e324: 8 total, 8 up, 8 in 2024-09-10T18:24:29.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:28 smithi099 ceph-mon[27037]: osdmap e324: 8 total, 8 up, 8 in 2024-09-10T18:24:29.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:28 smithi099 ceph-mon[37557]: osdmap e324: 8 total, 8 up, 8 in 2024-09-10T18:24:29.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:29 smithi133 ceph-mon[31214]: pgmap v449: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:29 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:29.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:29 smithi133 ceph-mon[31214]: osdmap e325: 8 total, 8 up, 8 in 2024-09-10T18:24:30.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:29 smithi099 ceph-mon[27037]: pgmap v449: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:30.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:29 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:30.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:29 smithi099 ceph-mon[27037]: osdmap e325: 8 total, 8 up, 8 in 2024-09-10T18:24:30.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:29 smithi099 ceph-mon[37557]: pgmap v449: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:30.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:29 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:30.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:29 smithi099 ceph-mon[37557]: osdmap e325: 8 total, 8 up, 8 in 2024-09-10T18:24:30.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:30 smithi133 ceph-mon[31214]: osdmap e326: 8 total, 8 up, 8 in 2024-09-10T18:24:30.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:30 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/194782689' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:30.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:30 smithi133 ceph-mon[31214]: from='client.35105 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:31.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:30 smithi099 ceph-mon[27037]: osdmap e326: 8 total, 8 up, 8 in 2024-09-10T18:24:31.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:30 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/194782689' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:31.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:30 smithi099 ceph-mon[27037]: from='client.35105 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:31.009 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:30 smithi099 ceph-mon[37557]: osdmap e326: 8 total, 8 up, 8 in 2024-09-10T18:24:31.009 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:30 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/194782689' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:31.009 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:30 smithi099 ceph-mon[37557]: from='client.35105 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:31.682 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_setxattr PASSED [ 83%] 2024-09-10T18:24:31.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:31 smithi133 ceph-mon[31214]: pgmap v452: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:31.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:31 smithi133 ceph-mon[31214]: from='client.35105 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:31.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:31 smithi133 ceph-mon[31214]: osdmap e327: 8 total, 8 up, 8 in 2024-09-10T18:24:32.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:31 smithi099 ceph-mon[27037]: pgmap v452: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:32.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:31 smithi099 ceph-mon[27037]: from='client.35105 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:32.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:31 smithi099 ceph-mon[27037]: osdmap e327: 8 total, 8 up, 8 in 2024-09-10T18:24:32.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:31 smithi099 ceph-mon[37557]: pgmap v452: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:32.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:31 smithi099 ceph-mon[37557]: from='client.35105 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:32.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:31 smithi099 ceph-mon[37557]: osdmap e327: 8 total, 8 up, 8 in 2024-09-10T18:24:32.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:32 smithi133 ceph-mon[31214]: osdmap e328: 8 total, 8 up, 8 in 2024-09-10T18:24:33.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:32 smithi099 ceph-mon[27037]: osdmap e328: 8 total, 8 up, 8 in 2024-09-10T18:24:33.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:32 smithi099 ceph-mon[37557]: osdmap e328: 8 total, 8 up, 8 in 2024-09-10T18:24:33.991 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:24:33 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:24:33.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:33 smithi133 ceph-mon[31214]: pgmap v455: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:33.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:33 smithi133 ceph-mon[31214]: osdmap e329: 8 total, 8 up, 8 in 2024-09-10T18:24:33.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:33 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:24:33.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:33 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-09-10T18:24:33.992 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:33 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-09-10T18:24:34.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:24:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:24:33] "GET /metrics HTTP/1.1" 200 225302 "" "Prometheus/2.43.0" 2024-09-10T18:24:34.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:33 smithi099 ceph-mon[27037]: pgmap v455: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:34.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:33 smithi099 ceph-mon[27037]: osdmap e329: 8 total, 8 up, 8 in 2024-09-10T18:24:34.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:33 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:24:34.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:33 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-09-10T18:24:34.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:33 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-09-10T18:24:34.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:33 smithi099 ceph-mon[37557]: pgmap v455: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:34.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:33 smithi099 ceph-mon[37557]: osdmap e329: 8 total, 8 up, 8 in 2024-09-10T18:24:34.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:33 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:24:34.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:33 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-09-10T18:24:34.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:33 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-09-10T18:24:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:34 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:34 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]': finished 2024-09-10T18:24:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:34 smithi133 ceph-mon[31214]: osdmap e330: 8 total, 8 up, 8 in 2024-09-10T18:24:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:34 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2"}]: dispatch 2024-09-10T18:24:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:34 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:24:34.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:34 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:24:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:34 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:35.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:34 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]': finished 2024-09-10T18:24:35.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:34 smithi099 ceph-mon[27037]: osdmap e330: 8 total, 8 up, 8 in 2024-09-10T18:24:35.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:34 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2"}]: dispatch 2024-09-10T18:24:35.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:34 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:24:35.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:34 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:24:35.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:34 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:35.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:34 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]': finished 2024-09-10T18:24:35.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:34 smithi099 ceph-mon[37557]: osdmap e330: 8 total, 8 up, 8 in 2024-09-10T18:24:35.007 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:34 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2"}]: dispatch 2024-09-10T18:24:35.008 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:34 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:24:35.008 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:34 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:24:35.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:35 smithi133 ceph-mon[31214]: pgmap v458: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:35.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:35 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:24:35.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:35 smithi133 ceph-mon[31214]: osdmap e331: 8 total, 8 up, 8 in 2024-09-10T18:24:35.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:35 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"dne","key":"key","value":"key"}]: dispatch 2024-09-10T18:24:35.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:35 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-09-10T18:24:35.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:35 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-09-10T18:24:36.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:35 smithi099 ceph-mon[27037]: pgmap v458: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:36.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:35 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:24:36.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:35 smithi099 ceph-mon[27037]: osdmap e331: 8 total, 8 up, 8 in 2024-09-10T18:24:36.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:35 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"dne","key":"key","value":"key"}]: dispatch 2024-09-10T18:24:36.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:35 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-09-10T18:24:36.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:35 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-09-10T18:24:36.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:35 smithi099 ceph-mon[37557]: pgmap v458: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:36.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:35 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:24:36.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:35 smithi099 ceph-mon[37557]: osdmap e331: 8 total, 8 up, 8 in 2024-09-10T18:24:36.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:35 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"dne","key":"key","value":"key"}]: dispatch 2024-09-10T18:24:36.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:35 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-09-10T18:24:36.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:35 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-09-10T18:24:36.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:36 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]': finished 2024-09-10T18:24:36.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:36 smithi133 ceph-mon[31214]: osdmap e332: 8 total, 8 up, 8 in 2024-09-10T18:24:36.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:36 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-09-10T18:24:36.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:36 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-09-10T18:24:37.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:36 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]': finished 2024-09-10T18:24:37.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:36 smithi099 ceph-mon[27037]: osdmap e332: 8 total, 8 up, 8 in 2024-09-10T18:24:37.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:36 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-09-10T18:24:37.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:36 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-09-10T18:24:37.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:36 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]': finished 2024-09-10T18:24:37.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:36 smithi099 ceph-mon[37557]: osdmap e332: 8 total, 8 up, 8 in 2024-09-10T18:24:37.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:36 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-09-10T18:24:37.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:36 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-09-10T18:24:37.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:24:37 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:24:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:24:37.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:37 smithi133 ceph-mon[31214]: pgmap v461: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:37.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:37 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]': finished 2024-09-10T18:24:37.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:37 smithi133 ceph-mon[31214]: osdmap e333: 8 total, 8 up, 8 in 2024-09-10T18:24:37.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:37 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-09-10T18:24:37.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:37 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-09-10T18:24:37.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:37 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:38.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:37 smithi099 ceph-mon[27037]: pgmap v461: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:38.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:37 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]': finished 2024-09-10T18:24:38.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:37 smithi099 ceph-mon[27037]: osdmap e333: 8 total, 8 up, 8 in 2024-09-10T18:24:38.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:37 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-09-10T18:24:38.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:37 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-09-10T18:24:38.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:37 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:38.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:37 smithi099 ceph-mon[37557]: pgmap v461: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:38.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:37 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]': finished 2024-09-10T18:24:38.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:37 smithi099 ceph-mon[37557]: osdmap e333: 8 total, 8 up, 8 in 2024-09-10T18:24:38.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:37 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-09-10T18:24:38.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:37 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-09-10T18:24:38.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:37 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:38.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:38 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]': finished 2024-09-10T18:24:38.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:38 smithi133 ceph-mon[31214]: osdmap e334: 8 total, 8 up, 8 in 2024-09-10T18:24:38.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:38 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:24:38.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:38 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:24:39.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:38 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]': finished 2024-09-10T18:24:39.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:38 smithi099 ceph-mon[27037]: osdmap e334: 8 total, 8 up, 8 in 2024-09-10T18:24:39.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:38 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:24:39.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:38 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:24:39.009 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:38 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]': finished 2024-09-10T18:24:39.009 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:38 smithi099 ceph-mon[37557]: osdmap e334: 8 total, 8 up, 8 in 2024-09-10T18:24:39.009 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:38 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:24:39.009 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:38 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:24:39.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:39 smithi133 ceph-mon[31214]: pgmap v464: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:39.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:39 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]': finished 2024-09-10T18:24:39.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:39 smithi133 ceph-mon[31214]: osdmap e335: 8 total, 8 up, 8 in 2024-09-10T18:24:39.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:39 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:39.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:39 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:40.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:39 smithi099 ceph-mon[27037]: pgmap v464: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:40.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:39 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]': finished 2024-09-10T18:24:40.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:39 smithi099 ceph-mon[27037]: osdmap e335: 8 total, 8 up, 8 in 2024-09-10T18:24:40.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:39 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:40.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:39 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:40.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:39 smithi099 ceph-mon[37557]: pgmap v464: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:40.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:39 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]': finished 2024-09-10T18:24:40.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:39 smithi099 ceph-mon[37557]: osdmap e335: 8 total, 8 up, 8 in 2024-09-10T18:24:40.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:39 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3488611333' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:40.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:39 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:40.753 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_applications PASSED [ 84%] 2024-09-10T18:24:41.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:40 smithi099 ceph-mon[27037]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:41.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:40 smithi099 ceph-mon[27037]: osdmap e336: 8 total, 8 up, 8 in 2024-09-10T18:24:41.005 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:40 smithi099 ceph-mon[37557]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:41.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:40 smithi099 ceph-mon[37557]: osdmap e336: 8 total, 8 up, 8 in 2024-09-10T18:24:41.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:40 smithi133 ceph-mon[31214]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:41.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:40 smithi133 ceph-mon[31214]: osdmap e336: 8 total, 8 up, 8 in 2024-09-10T18:24:42.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:41 smithi133 ceph-mon[31214]: pgmap v467: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:42.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:41 smithi133 ceph-mon[31214]: osdmap e337: 8 total, 8 up, 8 in 2024-09-10T18:24:42.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:41 smithi099 ceph-mon[27037]: pgmap v467: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:42.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:41 smithi099 ceph-mon[27037]: osdmap e337: 8 total, 8 up, 8 in 2024-09-10T18:24:42.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:41 smithi099 ceph-mon[37557]: pgmap v467: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:42.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:41 smithi099 ceph-mon[37557]: osdmap e337: 8 total, 8 up, 8 in 2024-09-10T18:24:43.142 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:42 smithi133 ceph-mon[31214]: osdmap e338: 8 total, 8 up, 8 in 2024-09-10T18:24:43.142 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:42 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1030971687' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:43.142 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:42 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1030971687' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:43.142 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:42 smithi133 ceph-mon[31214]: osdmap e339: 8 total, 8 up, 8 in 2024-09-10T18:24:43.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:42 smithi099 ceph-mon[27037]: osdmap e338: 8 total, 8 up, 8 in 2024-09-10T18:24:43.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:42 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1030971687' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:43.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:42 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1030971687' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:43.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:42 smithi099 ceph-mon[27037]: osdmap e339: 8 total, 8 up, 8 in 2024-09-10T18:24:43.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:42 smithi099 ceph-mon[37557]: osdmap e338: 8 total, 8 up, 8 in 2024-09-10T18:24:43.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:42 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1030971687' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:43.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:42 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1030971687' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:43.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:42 smithi099 ceph-mon[37557]: osdmap e339: 8 total, 8 up, 8 in 2024-09-10T18:24:43.759 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_service_daemon PASSED [ 85%] 2024-09-10T18:24:43.990 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:43 smithi133 ceph-mon[31214]: pgmap v470: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:43.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:43 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:43.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:43 smithi133 ceph-mon[31214]: osdmap e340: 8 total, 8 up, 8 in 2024-09-10T18:24:43.991 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:24:43 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:24:44.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:43 smithi099 ceph-mon[27037]: pgmap v470: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:44.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:43 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:44.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:43 smithi099 ceph-mon[27037]: osdmap e340: 8 total, 8 up, 8 in 2024-09-10T18:24:44.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:24:43 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:24:43] "GET /metrics HTTP/1.1" 200 225316 "" "Prometheus/2.43.0" 2024-09-10T18:24:44.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:43 smithi099 ceph-mon[37557]: pgmap v470: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:44.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:43 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:44.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:43 smithi099 ceph-mon[37557]: osdmap e340: 8 total, 8 up, 8 in 2024-09-10T18:24:45.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:44 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:45.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:44 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:45.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:44 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:46.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:45 smithi133 ceph-mon[31214]: pgmap v473: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:46.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:45 smithi133 ceph-mon[31214]: osdmap e341: 8 total, 8 up, 8 in 2024-09-10T18:24:46.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:45 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3725104752' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:46.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:45 smithi099 ceph-mon[27037]: pgmap v473: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:46.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:45 smithi099 ceph-mon[27037]: osdmap e341: 8 total, 8 up, 8 in 2024-09-10T18:24:46.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:45 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3725104752' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:46.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:45 smithi099 ceph-mon[37557]: pgmap v473: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:46.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:45 smithi099 ceph-mon[37557]: osdmap e341: 8 total, 8 up, 8 in 2024-09-10T18:24:46.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:45 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3725104752' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:46.792 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_alignment PASSED [ 86%] 2024-09-10T18:24:47.171 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:46 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3725104752' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:47.171 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:46 smithi133 ceph-mon[31214]: osdmap e342: 8 total, 8 up, 8 in 2024-09-10T18:24:47.171 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:46 smithi133 ceph-mon[31214]: osdmap e343: 8 total, 8 up, 8 in 2024-09-10T18:24:47.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:46 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3725104752' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:47.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:46 smithi099 ceph-mon[27037]: osdmap e342: 8 total, 8 up, 8 in 2024-09-10T18:24:47.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:46 smithi099 ceph-mon[27037]: osdmap e343: 8 total, 8 up, 8 in 2024-09-10T18:24:47.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:46 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3725104752' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:47.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:46 smithi099 ceph-mon[37557]: osdmap e342: 8 total, 8 up, 8 in 2024-09-10T18:24:47.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:46 smithi099 ceph-mon[37557]: osdmap e343: 8 total, 8 up, 8 in 2024-09-10T18:24:47.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:24:47 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:24:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:24:48.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:47 smithi133 ceph-mon[31214]: pgmap v476: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:48.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:47 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/4255186908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:24:48.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:47 smithi133 ceph-mon[31214]: from='client.35129 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:24:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:47 smithi099 ceph-mon[27037]: pgmap v476: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:47 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/4255186908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:24:48.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:47 smithi099 ceph-mon[27037]: from='client.35129 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:24:48.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:47 smithi099 ceph-mon[37557]: pgmap v476: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:47 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/4255186908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:24:48.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:47 smithi099 ceph-mon[37557]: from='client.35129 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:24:49.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:48 smithi133 ceph-mon[31214]: from='client.35129 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:24:49.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:48 smithi133 ceph-mon[31214]: osdmap e344: 8 total, 8 up, 8 in 2024-09-10T18:24:49.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:48 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/4255186908' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]: dispatch 2024-09-10T18:24:49.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:48 smithi133 ceph-mon[31214]: from='client.35129 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]: dispatch 2024-09-10T18:24:49.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:48 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:24:49.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:48 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:49.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:48 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[27037]: from='client.35129 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:24:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[27037]: osdmap e344: 8 total, 8 up, 8 in 2024-09-10T18:24:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/4255186908' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]: dispatch 2024-09-10T18:24:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[27037]: from='client.35129 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]: dispatch 2024-09-10T18:24:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:24:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:49.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:49.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[37557]: from='client.35129 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:24:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[37557]: osdmap e344: 8 total, 8 up, 8 in 2024-09-10T18:24:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/4255186908' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]: dispatch 2024-09-10T18:24:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[37557]: from='client.35129 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]: dispatch 2024-09-10T18:24:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:24:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:49.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:48 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:49 smithi133 ceph-mon[31214]: pgmap v479: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:49 smithi133 ceph-mon[31214]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:49 smithi133 ceph-mon[31214]: osdmap e345: 8 total, 8 up, 8 in 2024-09-10T18:24:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:49 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:24:50.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:49 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:24:50.242 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:49 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:49 smithi099 ceph-mon[27037]: pgmap v479: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:49 smithi099 ceph-mon[27037]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:49 smithi099 ceph-mon[27037]: osdmap e345: 8 total, 8 up, 8 in 2024-09-10T18:24:50.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:49 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:24:50.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:49 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:24:50.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:49 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:50.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:49 smithi099 ceph-mon[37557]: pgmap v479: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:50.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:49 smithi099 ceph-mon[37557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:50.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:49 smithi099 ceph-mon[37557]: osdmap e345: 8 total, 8 up, 8 in 2024-09-10T18:24:50.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:49 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:24:50.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:49 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:24:50.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:49 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' 2024-09-10T18:24:51.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:50 smithi133 ceph-mon[31214]: from='client.35129 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]': finished 2024-09-10T18:24:51.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:50 smithi133 ceph-mon[31214]: osdmap e346: 8 total, 8 up, 8 in 2024-09-10T18:24:51.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:50 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/4255186908' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:51.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:50 smithi133 ceph-mon[31214]: from='client.35129 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:51.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:50 smithi133 ceph-mon[31214]: from='client.35129 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:51.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:50 smithi133 ceph-mon[31214]: osdmap e347: 8 total, 8 up, 8 in 2024-09-10T18:24:51.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:50 smithi099 ceph-mon[27037]: from='client.35129 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]': finished 2024-09-10T18:24:51.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:50 smithi099 ceph-mon[27037]: osdmap e346: 8 total, 8 up, 8 in 2024-09-10T18:24:51.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:50 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/4255186908' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:51.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:50 smithi099 ceph-mon[27037]: from='client.35129 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:51.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:50 smithi099 ceph-mon[27037]: from='client.35129 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:51.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:50 smithi099 ceph-mon[27037]: osdmap e347: 8 total, 8 up, 8 in 2024-09-10T18:24:51.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:50 smithi099 ceph-mon[37557]: from='client.35129 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]': finished 2024-09-10T18:24:51.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:50 smithi099 ceph-mon[37557]: osdmap e346: 8 total, 8 up, 8 in 2024-09-10T18:24:51.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:50 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/4255186908' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:51.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:50 smithi099 ceph-mon[37557]: from='client.35129 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:51.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:50 smithi099 ceph-mon[37557]: from='client.35129 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:51.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:50 smithi099 ceph-mon[37557]: osdmap e347: 8 total, 8 up, 8 in 2024-09-10T18:24:51.828 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctxEc::test_alignment PASSED [ 87%] 2024-09-10T18:24:52.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:51 smithi133 ceph-mon[31214]: pgmap v482: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:52.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:51 smithi133 ceph-mon[31214]: osdmap e348: 8 total, 8 up, 8 in 2024-09-10T18:24:52.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:51 smithi099 ceph-mon[27037]: pgmap v482: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:52.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:51 smithi099 ceph-mon[27037]: osdmap e348: 8 total, 8 up, 8 in 2024-09-10T18:24:52.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:51 smithi099 ceph-mon[37557]: pgmap v482: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:52.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:51 smithi099 ceph-mon[37557]: osdmap e348: 8 total, 8 up, 8 in 2024-09-10T18:24:53.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:52 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:53.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:52 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:53.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:52 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:24:53.991 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:24:53 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:24:53.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:53 smithi133 ceph-mon[31214]: pgmap v485: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:53.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:53 smithi133 ceph-mon[31214]: osdmap e349: 8 total, 8 up, 8 in 2024-09-10T18:24:53.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:53 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3563452966' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:53.991 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:53 smithi133 ceph-mon[31214]: from='client.35135 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:54.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:24:53 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:24:53] "GET /metrics HTTP/1.1" 200 222952 "" "Prometheus/2.43.0" 2024-09-10T18:24:54.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:53 smithi099 ceph-mon[27037]: pgmap v485: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:54.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:53 smithi099 ceph-mon[27037]: osdmap e349: 8 total, 8 up, 8 in 2024-09-10T18:24:54.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:53 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3563452966' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:54.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:53 smithi099 ceph-mon[27037]: from='client.35135 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:54.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:53 smithi099 ceph-mon[37557]: pgmap v485: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:24:54.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:53 smithi099 ceph-mon[37557]: osdmap e349: 8 total, 8 up, 8 in 2024-09-10T18:24:54.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:53 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3563452966' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:54.006 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:53 smithi099 ceph-mon[37557]: from='client.35135 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:54.867 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx2::test_get_last_version PASSED [ 88%] 2024-09-10T18:24:55.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:54 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:55.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:54 smithi133 ceph-mon[31214]: from='client.35135 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:55.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:54 smithi133 ceph-mon[31214]: osdmap e350: 8 total, 8 up, 8 in 2024-09-10T18:24:55.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:54 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:55.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:54 smithi099 ceph-mon[27037]: from='client.35135 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:55.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:54 smithi099 ceph-mon[27037]: osdmap e350: 8 total, 8 up, 8 in 2024-09-10T18:24:55.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:54 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:24:55.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:54 smithi099 ceph-mon[37557]: from='client.35135 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:55.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:54 smithi099 ceph-mon[37557]: osdmap e350: 8 total, 8 up, 8 in 2024-09-10T18:24:56.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:55 smithi133 ceph-mon[31214]: pgmap v488: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:56.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:55 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:56.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:55 smithi133 ceph-mon[31214]: osdmap e351: 8 total, 8 up, 8 in 2024-09-10T18:24:56.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:55 smithi099 ceph-mon[27037]: pgmap v488: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:56.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:55 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:56.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:55 smithi099 ceph-mon[27037]: osdmap e351: 8 total, 8 up, 8 in 2024-09-10T18:24:56.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:55 smithi099 ceph-mon[37557]: pgmap v488: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:56.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:55 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:24:56.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:55 smithi099 ceph-mon[37557]: osdmap e351: 8 total, 8 up, 8 in 2024-09-10T18:24:57.171 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:56 smithi133 ceph-mon[31214]: osdmap e352: 8 total, 8 up, 8 in 2024-09-10T18:24:57.171 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:56 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1170870689' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:57.171 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:56 smithi133 ceph-mon[31214]: from='client.35141 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:57.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:56 smithi099 ceph-mon[27037]: osdmap e352: 8 total, 8 up, 8 in 2024-09-10T18:24:57.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:56 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1170870689' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:57.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:56 smithi099 ceph-mon[27037]: from='client.35141 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:57.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:56 smithi099 ceph-mon[37557]: osdmap e352: 8 total, 8 up, 8 in 2024-09-10T18:24:57.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:56 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1170870689' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:57.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:56 smithi099 ceph-mon[37557]: from='client.35141 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:24:57.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:24:57 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:24:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:24:57.883 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx2::test_get_stats PASSED [ 89%] 2024-09-10T18:24:58.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:57 smithi133 ceph-mon[31214]: pgmap v491: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:58.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:57 smithi133 ceph-mon[31214]: from='client.35141 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:58.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:57 smithi133 ceph-mon[31214]: osdmap e353: 8 total, 8 up, 8 in 2024-09-10T18:24:58.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:57 smithi099 ceph-mon[27037]: pgmap v491: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:58.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:57 smithi099 ceph-mon[27037]: from='client.35141 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:58.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:57 smithi099 ceph-mon[27037]: osdmap e353: 8 total, 8 up, 8 in 2024-09-10T18:24:58.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:57 smithi099 ceph-mon[37557]: pgmap v491: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:24:58.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:57 smithi099 ceph-mon[37557]: from='client.35141 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:24:58.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:57 smithi099 ceph-mon[37557]: osdmap e353: 8 total, 8 up, 8 in 2024-09-10T18:24:59.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:58 smithi133 ceph-mon[31214]: osdmap e354: 8 total, 8 up, 8 in 2024-09-10T18:24:59.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:58 smithi099 ceph-mon[27037]: osdmap e354: 8 total, 8 up, 8 in 2024-09-10T18:24:59.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:58 smithi099 ceph-mon[37557]: osdmap e354: 8 total, 8 up, 8 in 2024-09-10T18:25:00.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:59 smithi133 ceph-mon[31214]: pgmap v494: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:00.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:24:59 smithi133 ceph-mon[31214]: osdmap e355: 8 total, 8 up, 8 in 2024-09-10T18:25:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:59 smithi099 ceph-mon[27037]: pgmap v494: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:00.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:24:59 smithi099 ceph-mon[27037]: osdmap e355: 8 total, 8 up, 8 in 2024-09-10T18:25:00.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:59 smithi099 ceph-mon[37557]: pgmap v494: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:00.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:24:59 smithi099 ceph-mon[37557]: osdmap e355: 8 total, 8 up, 8 in 2024-09-10T18:25:00.920 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_read PASSED [ 91%] 2024-09-10T18:25:01.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:00 smithi133 ceph-mon[31214]: osdmap e356: 8 total, 8 up, 8 in 2024-09-10T18:25:01.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:00 smithi099 ceph-mon[27037]: osdmap e356: 8 total, 8 up, 8 in 2024-09-10T18:25:01.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:00 smithi099 ceph-mon[37557]: osdmap e356: 8 total, 8 up, 8 in 2024-09-10T18:25:02.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:01 smithi133 ceph-mon[31214]: pgmap v497: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:02.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:01 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:25:02.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:01 smithi133 ceph-mon[31214]: osdmap e357: 8 total, 8 up, 8 in 2024-09-10T18:25:02.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:01 smithi099 ceph-mon[27037]: pgmap v497: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:02.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:01 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:25:02.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:01 smithi099 ceph-mon[27037]: osdmap e357: 8 total, 8 up, 8 in 2024-09-10T18:25:02.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:01 smithi099 ceph-mon[37557]: pgmap v497: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:02.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:01 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:25:02.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:01 smithi099 ceph-mon[37557]: osdmap e357: 8 total, 8 up, 8 in 2024-09-10T18:25:03.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:02 smithi133 ceph-mon[31214]: osdmap e358: 8 total, 8 up, 8 in 2024-09-10T18:25:03.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:02 smithi099 ceph-mon[27037]: osdmap e358: 8 total, 8 up, 8 in 2024-09-10T18:25:03.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:02 smithi099 ceph-mon[37557]: osdmap e358: 8 total, 8 up, 8 in 2024-09-10T18:25:03.933 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:03 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:25:03] "GET /metrics HTTP/1.1" 200 222952 "" "Prometheus/2.43.0" 2024-09-10T18:25:03.933 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:25:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:25:03.953 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_seek PASSED [ 92%] 2024-09-10T18:25:04.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:03 smithi133 ceph-mon[31214]: pgmap v500: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:25:04.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:03 smithi133 ceph-mon[31214]: osdmap e359: 8 total, 8 up, 8 in 2024-09-10T18:25:04.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:03 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:04.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:03 smithi099 ceph-mon[37557]: pgmap v500: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:25:04.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:03 smithi099 ceph-mon[37557]: osdmap e359: 8 total, 8 up, 8 in 2024-09-10T18:25:04.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:03 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:04.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:03 smithi099 ceph-mon[27037]: pgmap v500: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:25:04.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:03 smithi099 ceph-mon[27037]: osdmap e359: 8 total, 8 up, 8 in 2024-09-10T18:25:04.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:03 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:05.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:04 smithi133 ceph-mon[31214]: osdmap e360: 8 total, 8 up, 8 in 2024-09-10T18:25:05.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:04 smithi133 ceph-mon[31214]: pgmap v503: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:05.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:04 smithi099 ceph-mon[27037]: osdmap e360: 8 total, 8 up, 8 in 2024-09-10T18:25:05.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:04 smithi099 ceph-mon[27037]: pgmap v503: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:05.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:04 smithi099 ceph-mon[37557]: osdmap e360: 8 total, 8 up, 8 in 2024-09-10T18:25:05.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:04 smithi099 ceph-mon[37557]: pgmap v503: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:06.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:05 smithi133 ceph-mon[31214]: osdmap e361: 8 total, 8 up, 8 in 2024-09-10T18:25:06.256 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:05 smithi099 ceph-mon[27037]: osdmap e361: 8 total, 8 up, 8 in 2024-09-10T18:25:06.257 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:05 smithi099 ceph-mon[37557]: osdmap e361: 8 total, 8 up, 8 in 2024-09-10T18:25:06.956 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_write PASSED [ 93%] 2024-09-10T18:25:07.241 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:25:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:25:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:25:07.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:06 smithi133 ceph-mon[31214]: osdmap e362: 8 total, 8 up, 8 in 2024-09-10T18:25:07.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:06 smithi133 ceph-mon[31214]: pgmap v506: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:07.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:06 smithi133 ceph-mon[31214]: osdmap e363: 8 total, 8 up, 8 in 2024-09-10T18:25:07.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:06 smithi099 ceph-mon[27037]: osdmap e362: 8 total, 8 up, 8 in 2024-09-10T18:25:07.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:06 smithi099 ceph-mon[27037]: pgmap v506: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:07.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:06 smithi099 ceph-mon[27037]: osdmap e363: 8 total, 8 up, 8 in 2024-09-10T18:25:07.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:06 smithi099 ceph-mon[37557]: osdmap e362: 8 total, 8 up, 8 in 2024-09-10T18:25:07.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:06 smithi099 ceph-mon[37557]: pgmap v506: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:07.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:06 smithi099 ceph-mon[37557]: osdmap e363: 8 total, 8 up, 8 in 2024-09-10T18:25:08.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:07 smithi133 ceph-mon[31214]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:25:08.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:07 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:08.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:07 smithi099 ceph-mon[27037]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:25:08.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:07 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:08.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:07 smithi099 ceph-mon[37557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:25:08.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:07 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:09.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:08 smithi133 ceph-mon[31214]: osdmap e364: 8 total, 8 up, 8 in 2024-09-10T18:25:09.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:08 smithi133 ceph-mon[31214]: pgmap v509: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:09.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:08 smithi099 ceph-mon[27037]: osdmap e364: 8 total, 8 up, 8 in 2024-09-10T18:25:09.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:08 smithi099 ceph-mon[27037]: pgmap v509: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:09.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:08 smithi099 ceph-mon[37557]: osdmap e364: 8 total, 8 up, 8 in 2024-09-10T18:25:09.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:08 smithi099 ceph-mon[37557]: pgmap v509: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:10.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:09 smithi133 ceph-mon[31214]: osdmap e365: 8 total, 8 up, 8 in 2024-09-10T18:25:10.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:09 smithi099 ceph-mon[27037]: osdmap e365: 8 total, 8 up, 8 in 2024-09-10T18:25:10.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:09 smithi099 ceph-mon[37557]: osdmap e365: 8 total, 8 up, 8 in 2024-09-10T18:25:11.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:10 smithi099 ceph-mon[27037]: osdmap e366: 8 total, 8 up, 8 in 2024-09-10T18:25:11.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:10 smithi099 ceph-mon[27037]: pgmap v512: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:11.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:10 smithi099 ceph-mon[37557]: osdmap e366: 8 total, 8 up, 8 in 2024-09-10T18:25:11.256 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:10 smithi099 ceph-mon[37557]: pgmap v512: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:11.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:10 smithi133 ceph-mon[31214]: osdmap e366: 8 total, 8 up, 8 in 2024-09-10T18:25:11.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:10 smithi133 ceph-mon[31214]: pgmap v512: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:12.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:11 smithi099 ceph-mon[27037]: osdmap e367: 8 total, 8 up, 8 in 2024-09-10T18:25:12.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:11 smithi099 ceph-mon[37557]: osdmap e367: 8 total, 8 up, 8 in 2024-09-10T18:25:12.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:11 smithi133 ceph-mon[31214]: osdmap e367: 8 total, 8 up, 8 in 2024-09-10T18:25:13.392 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:13 smithi133 ceph-mon[31214]: osdmap e368: 8 total, 8 up, 8 in 2024-09-10T18:25:13.392 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:13 smithi133 ceph-mon[31214]: pgmap v515: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:25:13.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:13 smithi099 ceph-mon[27037]: osdmap e368: 8 total, 8 up, 8 in 2024-09-10T18:25:13.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:13 smithi099 ceph-mon[27037]: pgmap v515: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:25:13.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:13 smithi099 ceph-mon[37557]: osdmap e368: 8 total, 8 up, 8 in 2024-09-10T18:25:13.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:13 smithi099 ceph-mon[37557]: pgmap v515: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:25:13.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:25:13 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:25:14.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:13 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:25:13] "GET /metrics HTTP/1.1" 200 222990 "" "Prometheus/2.43.0" 2024-09-10T18:25:14.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:14 smithi133 ceph-mon[31214]: osdmap e369: 8 total, 8 up, 8 in 2024-09-10T18:25:14.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:14 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1700457270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:25:14.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:14 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:14.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:14 smithi099 ceph-mon[27037]: osdmap e369: 8 total, 8 up, 8 in 2024-09-10T18:25:14.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:14 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1700457270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:25:14.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:14 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:14.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:14 smithi099 ceph-mon[37557]: osdmap e369: 8 total, 8 up, 8 in 2024-09-10T18:25:14.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:14 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1700457270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-10T18:25:14.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:14 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:15.020 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoCtxSelfManagedSnaps::test PASSED [ 94%] 2024-09-10T18:25:15.043 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_monmap_dump PASSED [ 95%] 2024-09-10T18:25:15.059 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_osd_bench PASSED [ 96%] 2024-09-10T18:25:15.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:15 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/1700457270' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:25:15.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:15 smithi133 ceph-mon[31214]: osdmap e370: 8 total, 8 up, 8 in 2024-09-10T18:25:15.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:15 smithi133 ceph-mon[31214]: pgmap v518: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:25:15.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:15 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/1700457270' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:25:15.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:15 smithi099 ceph-mon[27037]: osdmap e370: 8 total, 8 up, 8 in 2024-09-10T18:25:15.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:15 smithi099 ceph-mon[27037]: pgmap v518: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:25:15.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:15 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/1700457270' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-10T18:25:15.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:15 smithi099 ceph-mon[37557]: osdmap e370: 8 total, 8 up, 8 in 2024-09-10T18:25:15.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:15 smithi099 ceph-mon[37557]: pgmap v518: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:25:16.050 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_ceph_osd_pool_create_utf8 PASSED [ 97%] 2024-09-10T18:25:16.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:16 smithi133 ceph-mon[31214]: osdmap e371: 8 total, 8 up, 8 in 2024-09-10T18:25:16.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:16 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2223276892' entity='client.admin' cmd=[{"prefix": "mon dump"}]: dispatch 2024-09-10T18:25:16.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:16 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2223276892' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:25:16.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:16 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/2223276892' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json", "epoch": 1003}]: dispatch 2024-09-10T18:25:16.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:16 smithi133 ceph-mon[31214]: from='client.? v1:172.21.15.99:0/3503786099' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-09-10T18:25:16.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:16 smithi133 ceph-mon[31214]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-09-10T18:25:16.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:16 smithi099 ceph-mon[27037]: osdmap e371: 8 total, 8 up, 8 in 2024-09-10T18:25:16.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:16 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2223276892' entity='client.admin' cmd=[{"prefix": "mon dump"}]: dispatch 2024-09-10T18:25:16.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:16 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2223276892' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:25:16.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:16 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/2223276892' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json", "epoch": 1003}]: dispatch 2024-09-10T18:25:16.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:16 smithi099 ceph-mon[27037]: from='client.? v1:172.21.15.99:0/3503786099' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-09-10T18:25:16.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:16 smithi099 ceph-mon[27037]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-09-10T18:25:16.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:16 smithi099 ceph-mon[37557]: osdmap e371: 8 total, 8 up, 8 in 2024-09-10T18:25:16.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:16 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2223276892' entity='client.admin' cmd=[{"prefix": "mon dump"}]: dispatch 2024-09-10T18:25:16.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:16 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2223276892' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:25:16.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:16 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/2223276892' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json", "epoch": 1003}]: dispatch 2024-09-10T18:25:16.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:16 smithi099 ceph-mon[37557]: from='client.? v1:172.21.15.99:0/3503786099' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-09-10T18:25:16.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:16 smithi099 ceph-mon[37557]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-09-10T18:25:17.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:17 smithi133 ceph-mon[31214]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]': finished 2024-09-10T18:25:17.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:17 smithi133 ceph-mon[31214]: osdmap e372: 8 total, 8 up, 8 in 2024-09-10T18:25:17.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:17 smithi133 ceph-mon[31214]: pgmap v521: 180 pgs: 16 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:17.491 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:25:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:25:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:25:17.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:17 smithi099 ceph-mon[27037]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]': finished 2024-09-10T18:25:17.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:17 smithi099 ceph-mon[27037]: osdmap e372: 8 total, 8 up, 8 in 2024-09-10T18:25:17.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:17 smithi099 ceph-mon[27037]: pgmap v521: 180 pgs: 16 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:17.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:17 smithi099 ceph-mon[37557]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]': finished 2024-09-10T18:25:17.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:17 smithi099 ceph-mon[37557]: osdmap e372: 8 total, 8 up, 8 in 2024-09-10T18:25:17.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:17 smithi099 ceph-mon[37557]: pgmap v521: 180 pgs: 16 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:18.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:18 smithi133 ceph-mon[31214]: osdmap e373: 8 total, 8 up, 8 in 2024-09-10T18:25:18.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:18 smithi099 ceph-mon[27037]: osdmap e373: 8 total, 8 up, 8 in 2024-09-10T18:25:18.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:18 smithi099 ceph-mon[37557]: osdmap e373: 8 total, 8 up, 8 in 2024-09-10T18:25:19.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:19 smithi133 ceph-mon[31214]: osdmap e374: 8 total, 8 up, 8 in 2024-09-10T18:25:19.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:19 smithi133 ceph-mon[31214]: pgmap v524: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:25:19.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:19 smithi133 ceph-mon[31214]: osdmap e375: 8 total, 8 up, 8 in 2024-09-10T18:25:19.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:19 smithi099 ceph-mon[27037]: osdmap e374: 8 total, 8 up, 8 in 2024-09-10T18:25:19.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:19 smithi099 ceph-mon[27037]: pgmap v524: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:25:19.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:19 smithi099 ceph-mon[27037]: osdmap e375: 8 total, 8 up, 8 in 2024-09-10T18:25:19.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:19 smithi099 ceph-mon[37557]: osdmap e374: 8 total, 8 up, 8 in 2024-09-10T18:25:19.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:19 smithi099 ceph-mon[37557]: pgmap v524: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:25:19.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:19 smithi099 ceph-mon[37557]: osdmap e375: 8 total, 8 up, 8 in 2024-09-10T18:25:20.081 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestWatchNotify::test PASSED [ 98%] 2024-09-10T18:25:20.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:20 smithi133 ceph-mon[31214]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:25:20.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:20 smithi099 ceph-mon[27037]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:25:20.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:20 smithi099 ceph-mon[37557]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:25:21.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:21 smithi133 ceph-mon[31214]: osdmap e376: 8 total, 8 up, 8 in 2024-09-10T18:25:21.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:21 smithi133 ceph-mon[31214]: pgmap v527: 180 pgs: 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:21.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:21 smithi099 ceph-mon[27037]: osdmap e376: 8 total, 8 up, 8 in 2024-09-10T18:25:21.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:21 smithi099 ceph-mon[27037]: pgmap v527: 180 pgs: 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:21.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:21 smithi099 ceph-mon[37557]: osdmap e376: 8 total, 8 up, 8 in 2024-09-10T18:25:21.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:21 smithi099 ceph-mon[37557]: pgmap v527: 180 pgs: 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:25:22.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:22 smithi133 ceph-mon[31214]: osdmap e377: 8 total, 8 up, 8 in 2024-09-10T18:25:22.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:22 smithi099 ceph-mon[27037]: osdmap e377: 8 total, 8 up, 8 in 2024-09-10T18:25:22.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:22 smithi099 ceph-mon[37557]: osdmap e377: 8 total, 8 up, 8 in 2024-09-10T18:25:23.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:23 smithi133 ceph-mon[31214]: osdmap e378: 8 total, 8 up, 8 in 2024-09-10T18:25:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:23 smithi133 ceph-mon[31214]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:23.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:23 smithi133 ceph-mon[31214]: pgmap v530: 212 pgs: 32 unknown, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:25:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:23 smithi099 ceph-mon[27037]: osdmap e378: 8 total, 8 up, 8 in 2024-09-10T18:25:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:23 smithi099 ceph-mon[27037]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:23.505 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:23 smithi099 ceph-mon[27037]: pgmap v530: 212 pgs: 32 unknown, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:25:23.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:23 smithi099 ceph-mon[37557]: osdmap e378: 8 total, 8 up, 8 in 2024-09-10T18:25:23.505 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:23 smithi099 ceph-mon[37557]: from='mgr.34505 v1:172.21.15.99:0/3867071755' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:25:23.506 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:23 smithi099 ceph-mon[37557]: pgmap v530: 212 pgs: 32 unknown, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:25:23.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:25:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug there is no tcmu-runner data available 2024-09-10T18:25:24.005 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:25:23] "GET /metrics HTTP/1.1" 200 227696 "" "Prometheus/2.43.0" 2024-09-10T18:25:24.108 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestWatchNotify::test_aio_notify PASSED [100%] 2024-09-10T18:25:24.108 INFO:tasks.workunit.client.0.smithi099.stdout: 2024-09-10T18:25:24.108 INFO:tasks.workunit.client.0.smithi099.stdout:=============================== warnings summary =============================== 2024-09-10T18:25:24.108 INFO:tasks.workunit.client.0.smithi099.stdout:../../../clone.client.0/src/test/pybind/test_rados.py:210 2024-09-10T18:25:24.108 INFO:tasks.workunit.client.0.smithi099.stdout: /home/ubuntu/cephtest/clone.client.0/src/test/pybind/test_rados.py:210: DeprecationWarning: invalid escape sequence \- 2024-09-10T18:25:24.108 INFO:tasks.workunit.client.0.smithi099.stdout: assert re.match('[0-9a-f\-]{36}', fsid, re.I) 2024-09-10T18:25:24.108 INFO:tasks.workunit.client.0.smithi099.stdout: 2024-09-10T18:25:24.108 INFO:tasks.workunit.client.0.smithi099.stdout::210 2024-09-10T18:25:24.109 INFO:tasks.workunit.client.0.smithi099.stdout::210 2024-09-10T18:25:24.109 INFO:tasks.workunit.client.0.smithi099.stdout::210 2024-09-10T18:25:24.109 INFO:tasks.workunit.client.0.smithi099.stdout::210 2024-09-10T18:25:24.109 INFO:tasks.workunit.client.0.smithi099.stdout::210 2024-09-10T18:25:24.109 INFO:tasks.workunit.client.0.smithi099.stdout::210 2024-09-10T18:25:24.109 INFO:tasks.workunit.client.0.smithi099.stdout::210 2024-09-10T18:25:24.109 INFO:tasks.workunit.client.0.smithi099.stdout::210 2024-09-10T18:25:24.109 INFO:tasks.workunit.client.0.smithi099.stdout::210 2024-09-10T18:25:24.109 INFO:tasks.workunit.client.0.smithi099.stdout: :210: DeprecationWarning: invalid escape sequence \- 2024-09-10T18:25:24.109 INFO:tasks.workunit.client.0.smithi099.stdout: 2024-09-10T18:25:24.109 INFO:tasks.workunit.client.0.smithi099.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2024-09-10T18:25:24.109 INFO:tasks.workunit.client.0.smithi099.stdout:================= 89 passed, 10 warnings in 319.42s (0:05:19) ================== 2024-09-10T18:25:24.131 INFO:tasks.workunit.client.0.smithi099.stderr:+ exit 0 2024-09-10T18:25:24.132 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T18:25:24.132 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T18:25:24.161 INFO:tasks.workunit:Stopping ['rados/test_python.sh'] on client.0... 2024-09-10T18:25:24.161 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T18:25:24.480 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:24 smithi099 ceph-mon[27037]: osdmap e379: 8 total, 8 up, 8 in 2024-09-10T18:25:24.480 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:24 smithi099 ceph-mon[27037]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:24.481 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:24 smithi099 ceph-mon[37557]: osdmap e379: 8 total, 8 up, 8 in 2024-09-10T18:25:24.481 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:24 smithi099 ceph-mon[37557]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:24.482 DEBUG:teuthology.parallel:result is None 2024-09-10T18:25:24.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T18:25:24.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:24 smithi133 ceph-mon[31214]: osdmap e379: 8 total, 8 up, 8 in 2024-09-10T18:25:24.490 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:24 smithi133 ceph-mon[31214]: from='client.14553 v1:172.21.15.133:0/684458883' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:25:24.507 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T18:25:24.507 DEBUG:teuthology.orchestra.run.smithi099:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-10T18:25:24.580 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T18:25:24.580 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T18:25:24.589 INFO:tasks.cephadm:Teardown begin 2024-09-10T18:25:24.590 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:25:24.645 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:25:24.677 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T18:25:24.677 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:25:24.701 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:25:24.734 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T18:25:24.734 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T18:25:24.734 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.a 2024-09-10T18:25:25.049 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:24 smithi099 systemd[1]: Stopping Ceph mon.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:25:25.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:25 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a[27033]: 2024-09-10T18:25:25.047+0000 7fe73fcfb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:25:25.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:25 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a[27033]: 2024-09-10T18:25:25.047+0000 7fe73fcfb640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T18:25:25.506 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:25 smithi099 podman[76170]: 2024-09-10 18:25:25.221505464 +0000 UTC m=+0.293004399 container died 10e130d78a7bf7e9e0ab1d68276f964ff9f9c131219556bea1700efac9674144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T18:25:25.507 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:25 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:25:25] ENGINE Bus STOPPING 2024-09-10T18:25:25.507 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:25 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:25:25] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:25:25.507 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:25 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:25:25] ENGINE Bus STOPPED 2024-09-10T18:25:25.507 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:25 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:25:25] ENGINE Bus STARTING 2024-09-10T18:25:25.507 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:25 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:25:25] ENGINE Serving on http://:::9283 2024-09-10T18:25:25.507 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:25 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:25:25] ENGINE Bus STARTED 2024-09-10T18:25:25.952 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:25 smithi099 podman[76170]: 2024-09-10 18:25:25.740379546 +0000 UTC m=+0.811878469 container cleanup 10e130d78a7bf7e9e0ab1d68276f964ff9f9c131219556bea1700efac9674144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T18:25:25.952 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:25 smithi099 bash[76170]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a 2024-09-10T18:25:26.164 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T18:25:26.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:25 smithi099 podman[76193]: 2024-09-10 18:25:25.950323562 +0000 UTC m=+0.729328492 container remove 10e130d78a7bf7e9e0ab1d68276f964ff9f9c131219556bea1700efac9674144 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T18:25:26.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:26 smithi099 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T18:25:26.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:26 smithi099 systemd[1]: Stopped Ceph mon.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:25:26.255 INFO:journalctl@ceph.mon.a.smithi099.stdout:Sep 10 18:25:26 smithi099 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.a.service: Consumed 33.853s CPU time. 2024-09-10T18:25:26.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:25:26.314 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T18:25:26.314 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-10T18:25:26.314 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.c 2024-09-10T18:25:26.596 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:26 smithi099 systemd[1]: Stopping Ceph mon.c for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:25:26.878 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:26 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-c[37553]: 2024-09-10T18:25:26.594+0000 7fc197844640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:25:26.879 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:26 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-c[37553]: 2024-09-10T18:25:26.594+0000 7fc197844640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T18:25:26.879 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:26 smithi099 podman[76307]: 2024-09-10 18:25:26.76842606 +0000 UTC m=+0.276095715 container died 0ea6af5bbd90e3f8f9c8c2ba64b010eaca9444a98e2498e53ef3d6b508fd58ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:25:26.879 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:26 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:25:26] ENGINE Bus STOPPING 2024-09-10T18:25:26.879 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:26 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:25:26] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:25:26.879 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:26 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:25:26] ENGINE Bus STOPPED 2024-09-10T18:25:26.879 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:26 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:25:26] ENGINE Bus STARTING 2024-09-10T18:25:26.879 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:26 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:25:26] ENGINE Serving on http://:::9283 2024-09-10T18:25:26.879 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:26 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: [10/Sep/2024:18:25:26] ENGINE Bus STARTED 2024-09-10T18:25:27.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:27 smithi099 podman[76307]: 2024-09-10 18:25:27.111842551 +0000 UTC m=+0.619512209 container cleanup 0ea6af5bbd90e3f8f9c8c2ba64b010eaca9444a98e2498e53ef3d6b508fd58ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, io.buildah.version=1.37.2) 2024-09-10T18:25:27.255 INFO:journalctl@ceph.mon.c.smithi099.stdout:Sep 10 18:25:27 smithi099 bash[76307]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-c 2024-09-10T18:25:27.490 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:25:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x[33088]: ::ffff:172.21.15.133 - - [10/Sep/2024:18:25:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:25:27.548 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T18:25:27.582 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:25:27.582 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-10T18:25:27.582 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-10T18:25:27.582 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.b 2024-09-10T18:25:27.978 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:27 smithi133 systemd[1]: Stopping Ceph mon.b for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:25:28.240 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-b[31210]: 2024-09-10T18:25:27.975+0000 7f17586b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:25:28.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-b[31210]: 2024-09-10T18:25:27.975+0000 7f17586b3640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T18:25:28.241 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:28 smithi133 podman[65407]: 2024-09-10 18:25:28.159872465 +0000 UTC m=+0.375509167 container died dc0f0783b0838c574c61cb772c281c9f48525a08266ee53c617decc3dec5c730 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T18:25:28.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:28 smithi133 podman[65407]: 2024-09-10 18:25:28.617499865 +0000 UTC m=+0.833136555 container cleanup dc0f0783b0838c574c61cb772c281c9f48525a08266ee53c617decc3dec5c730 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T18:25:28.741 INFO:journalctl@ceph.mon.b.smithi133.stdout:Sep 10 18:25:28 smithi133 bash[65407]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mon-b 2024-09-10T18:25:29.050 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T18:25:29.093 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:25:29.093 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-10T18:25:29.093 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T18:25:29.093 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mgr.y 2024-09-10T18:25:29.505 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:29 smithi099 systemd[1]: Stopping Ceph mgr.y for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:25:29.802 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:29 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:25:29.507+0000 7f239ad16640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:25:29.803 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:29 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y[27263]: 2024-09-10T18:25:29.507+0000 7f239ad16640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T18:25:29.803 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:29 smithi099 podman[76442]: 2024-09-10 18:25:29.539342011 +0000 UTC m=+0.261418103 container died 8da925e9c2a9f22773cd5ce8f9b7f4a9aa98b8791e1655dbbf1b4dddbc25e4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T18:25:30.187 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:29 smithi099 podman[76442]: 2024-09-10 18:25:29.900744779 +0000 UTC m=+0.622820863 container cleanup 8da925e9c2a9f22773cd5ce8f9b7f4a9aa98b8791e1655dbbf1b4dddbc25e4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T18:25:30.187 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 10 18:25:29 smithi099 bash[76442]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-y 2024-09-10T18:25:30.329 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T18:25:30.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:25:30.363 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T18:25:30.363 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T18:25:30.364 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mgr.x 2024-09-10T18:25:30.686 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:25:30 smithi133 systemd[1]: Stopping Ceph mgr.x for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:25:30.991 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:25:30 smithi133 podman[65533]: 2024-09-10 18:25:30.684435981 +0000 UTC m=+0.121444787 container died 4c566f03fa35122fbaf31ea92ddd4265c8b5e5b740740a022d6e0ac0171c73f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T18:25:31.431 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:25:31 smithi133 podman[65533]: 2024-09-10 18:25:31.029988025 +0000 UTC m=+0.466996828 container cleanup 4c566f03fa35122fbaf31ea92ddd4265c8b5e5b740740a022d6e0ac0171c73f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T18:25:31.431 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:25:31 smithi133 bash[65533]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x 2024-09-10T18:25:31.431 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:25:31 smithi133 podman[65545]: 2024-09-10 18:25:31.223170303 +0000 UTC m=+0.534202291 container remove 4c566f03fa35122fbaf31ea92ddd4265c8b5e5b740740a022d6e0ac0171c73f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:25:31.431 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Sep 10 18:25:31 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:25:31.456 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T18:25:31.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:25:31.491 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T18:25:31.491 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T18:25:31.491 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.0 2024-09-10T18:25:31.806 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:31 smithi099 systemd[1]: Stopping Ceph osd.0 for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:25:32.255 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:31 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0[41432]: 2024-09-10T18:25:31.804+0000 7f1096773640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:25:32.255 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:31 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0[41432]: 2024-09-10T18:25:31.804+0000 7f1096773640 -1 osd.0 380 *** Got signal Terminated *** 2024-09-10T18:25:32.255 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:31 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0[41432]: 2024-09-10T18:25:31.804+0000 7f1096773640 -1 osd.0 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:25:37.557 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:37 smithi099 podman[76562]: 2024-09-10 18:25:37.29362338 +0000 UTC m=+5.614602030 container died 5e2a3bca744f233da1b8fb9caba835304831fc4b790f5a63d38a2d775a9aa59c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:25:37.833 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:37 smithi099 podman[76562]: 2024-09-10 18:25:37.638834639 +0000 UTC m=+5.959813288 container cleanup 5e2a3bca744f233da1b8fb9caba835304831fc4b790f5a63d38a2d775a9aa59c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:25:37.833 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:37 smithi099 bash[76562]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0 2024-09-10T18:25:38.225 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:37 smithi099 podman[76577]: 2024-09-10 18:25:37.831779468 +0000 UTC m=+0.531801842 container remove 5e2a3bca744f233da1b8fb9caba835304831fc4b790f5a63d38a2d775a9aa59c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2) 2024-09-10T18:25:38.505 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:38 smithi099 podman[76670]: 2024-09-10 18:25:38.224337823 +0000 UTC m=+0.090020702 container create a2d213c6c8b1bae1951740d4ced8f9687bad549a537ccaafbb154caf5b3e3a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T18:25:38.506 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:38 smithi099 podman[76670]: 2024-09-10 18:25:38.1498391 +0000 UTC m=+0.015521954 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:25:38.506 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:38 smithi099 podman[76670]: 2024-09-10 18:25:38.443398212 +0000 UTC m=+0.309081058 container init a2d213c6c8b1bae1951740d4ced8f9687bad549a537ccaafbb154caf5b3e3a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T18:25:38.506 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:38 smithi099 podman[76670]: 2024-09-10 18:25:38.4468287 +0000 UTC m=+0.312511551 container start a2d213c6c8b1bae1951740d4ced8f9687bad549a537ccaafbb154caf5b3e3a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True) 2024-09-10T18:25:38.982 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:38 smithi099 podman[76670]: 2024-09-10 18:25:38.560180714 +0000 UTC m=+0.425863555 container attach a2d213c6c8b1bae1951740d4ced8f9687bad549a537ccaafbb154caf5b3e3a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T18:25:38.982 INFO:journalctl@ceph.osd.0.smithi099.stdout:Sep 10 18:25:38 smithi099 podman[76670]: 2024-09-10 18:25:38.626714058 +0000 UTC m=+0.492396896 container died a2d213c6c8b1bae1951740d4ced8f9687bad549a537ccaafbb154caf5b3e3a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T18:25:39.306 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T18:25:39.348 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:25:39.348 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T18:25:39.348 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T18:25:39.348 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.1 2024-09-10T18:25:39.578 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:39 smithi099 systemd[1]: Stopping Ceph osd.1 for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:25:40.005 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:39 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1[47455]: 2024-09-10T18:25:39.727+0000 7ff43959b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:25:40.005 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:39 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1[47455]: 2024-09-10T18:25:39.727+0000 7ff43959b640 -1 osd.1 380 *** Got signal Terminated *** 2024-09-10T18:25:40.005 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:39 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1[47455]: 2024-09-10T18:25:39.727+0000 7ff43959b640 -1 osd.1 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:25:45.337 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:45 smithi099 podman[76774]: 2024-09-10 18:25:45.06411669 +0000 UTC m=+5.485398535 container died 042a6623c6b753b99ca01099b8424fcf78b6c8240637dff76f0386bcf4fd603a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:25:45.755 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:45 smithi099 podman[76774]: 2024-09-10 18:25:45.433717788 +0000 UTC m=+5.854999621 container cleanup 042a6623c6b753b99ca01099b8424fcf78b6c8240637dff76f0386bcf4fd603a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2) 2024-09-10T18:25:45.755 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:45 smithi099 bash[76774]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1 2024-09-10T18:25:46.102 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:45 smithi099 podman[76802]: 2024-09-10 18:25:45.77250913 +0000 UTC m=+0.707988785 container remove 042a6623c6b753b99ca01099b8424fcf78b6c8240637dff76f0386bcf4fd603a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:25:46.390 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:46 smithi099 podman[76894]: 2024-09-10 18:25:46.003579357 +0000 UTC m=+0.015715147 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:25:46.390 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:46 smithi099 podman[76894]: 2024-09-10 18:25:46.111564515 +0000 UTC m=+0.123700344 container create 0e5a44bb2f90cc06c902fad45788225b49bb1bd002df26f4201c29948a822bc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:25:46.755 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:46 smithi099 podman[76894]: 2024-09-10 18:25:46.389320999 +0000 UTC m=+0.401456788 container init 0e5a44bb2f90cc06c902fad45788225b49bb1bd002df26f4201c29948a822bc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:25:46.755 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:46 smithi099 podman[76894]: 2024-09-10 18:25:46.392672183 +0000 UTC m=+0.404807976 container start 0e5a44bb2f90cc06c902fad45788225b49bb1bd002df26f4201c29948a822bc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:25:46.756 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:46 smithi099 podman[76894]: 2024-09-10 18:25:46.472350347 +0000 UTC m=+0.484486132 container attach 0e5a44bb2f90cc06c902fad45788225b49bb1bd002df26f4201c29948a822bc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0) 2024-09-10T18:25:46.756 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 10 18:25:46 smithi099 podman[76894]: 2024-09-10 18:25:46.569838152 +0000 UTC m=+0.581973937 container died 0e5a44bb2f90cc06c902fad45788225b49bb1bd002df26f4201c29948a822bc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:25:47.201 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T18:25:47.241 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:25:47.241 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T18:25:47.242 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T18:25:47.242 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.2 2024-09-10T18:25:47.505 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:47 smithi099 systemd[1]: Stopping Ceph osd.2 for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:25:48.005 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:47 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2[53449]: 2024-09-10T18:25:47.553+0000 7effda508640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:25:48.005 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:47 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2[53449]: 2024-09-10T18:25:47.553+0000 7effda508640 -1 osd.2 380 *** Got signal Terminated *** 2024-09-10T18:25:48.005 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:47 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2[53449]: 2024-09-10T18:25:47.553+0000 7effda508640 -1 osd.2 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:25:53.188 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:52 smithi099 podman[76995]: 2024-09-10 18:25:52.861430358 +0000 UTC m=+5.400212911 container died 1df1b67e40bf986a744ab65bb858889a2ec54422d37cd191fddd0db739a0982b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-10T18:25:53.505 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:53 smithi099 podman[76995]: 2024-09-10 18:25:53.186519034 +0000 UTC m=+5.725301589 container cleanup 1df1b67e40bf986a744ab65bb858889a2ec54422d37cd191fddd0db739a0982b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:25:53.505 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:53 smithi099 bash[76995]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2 2024-09-10T18:25:53.505 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:53 smithi099 podman[77014]: 2024-09-10 18:25:53.387971791 +0000 UTC m=+0.520638682 container remove 1df1b67e40bf986a744ab65bb858889a2ec54422d37cd191fddd0db739a0982b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:25:53.802 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:25:53 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:25:53.529Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi099.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.99:8443: connect: connection refused" 2024-09-10T18:25:53.802 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:25:53 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:25:53.530Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi133.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.133:8443: connect: connection refused" 2024-09-10T18:25:54.059 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:53 smithi099 podman[77103]: 2024-09-10 18:25:53.703799116 +0000 UTC m=+0.015783977 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:25:54.059 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:53 smithi099 podman[77103]: 2024-09-10 18:25:53.830479793 +0000 UTC m=+0.142464686 container create 65e49052f1467ac7b41494d96ea6ff1267df28adbdf527d4d797f71b513cc21b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T18:25:54.059 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:54 smithi099 podman[77103]: 2024-09-10 18:25:54.057844828 +0000 UTC m=+0.369829682 container init 65e49052f1467ac7b41494d96ea6ff1267df28adbdf527d4d797f71b513cc21b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS) 2024-09-10T18:25:54.505 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:54 smithi099 podman[77103]: 2024-09-10 18:25:54.061043852 +0000 UTC m=+0.373028711 container start 65e49052f1467ac7b41494d96ea6ff1267df28adbdf527d4d797f71b513cc21b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T18:25:54.505 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:54 smithi099 podman[77103]: 2024-09-10 18:25:54.124389027 +0000 UTC m=+0.436373895 container attach 65e49052f1467ac7b41494d96ea6ff1267df28adbdf527d4d797f71b513cc21b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:25:54.505 INFO:journalctl@ceph.osd.2.smithi099.stdout:Sep 10 18:25:54 smithi099 podman[77103]: 2024-09-10 18:25:54.233435348 +0000 UTC m=+0.545420217 container died 65e49052f1467ac7b41494d96ea6ff1267df28adbdf527d4d797f71b513cc21b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:25:54.833 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T18:25:54.871 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:25:54.871 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T18:25:54.871 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T18:25:54.872 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.3 2024-09-10T18:25:55.200 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:25:54 smithi099 systemd[1]: Stopping Ceph osd.3 for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:25:55.505 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:25:55 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3[59505]: 2024-09-10T18:25:55.197+0000 7f319c00b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:25:55.505 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:25:55 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3[59505]: 2024-09-10T18:25:55.197+0000 7f319c00b640 -1 osd.3 380 *** Got signal Terminated *** 2024-09-10T18:25:55.505 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:25:55 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3[59505]: 2024-09-10T18:25:55.197+0000 7f319c00b640 -1 osd.3 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:25:59.083 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:25:58 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:25:58.830+0000 7fe090cb2640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:33.112560+0000 front 2024-09-10T18:25:33.112428+0000 (oldest deadline 2024-09-10T18:25:58.412477+0000) 2024-09-10T18:25:59.491 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:25:59 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:25:59.080+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:00.005 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:25:59 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3[59505]: 2024-09-10T18:25:59.703+0000 7f3195e1e640 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:33.354472+0000 front 2024-09-10T18:25:33.354397+0000 (oldest deadline 2024-09-10T18:25:59.254475+0000) 2024-09-10T18:26:00.095 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:25:59 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:25:59.995+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:00.095 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:25:59 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:25:59.838+0000 7fe090cb2640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:33.112560+0000 front 2024-09-10T18:25:33.112428+0000 (oldest deadline 2024-09-10T18:25:58.412477+0000) 2024-09-10T18:26:00.095 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:00.062+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:00.924 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:26:00 smithi099 podman[77204]: 2024-09-10 18:26:00.645618031 +0000 UTC m=+5.554725278 container died ddad0f843aa7e13fbedf86f7a47536cd80b5e618814e81e6130b4f3e4bc8b418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T18:26:01.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:01 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:01.019+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:01.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:01 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:01.010+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:01.241 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:00 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:00.885+0000 7fe090cb2640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:33.112560+0000 front 2024-09-10T18:25:33.112428+0000 (oldest deadline 2024-09-10T18:25:58.412477+0000) 2024-09-10T18:26:01.255 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:26:01 smithi099 podman[77204]: 2024-09-10 18:26:01.005074056 +0000 UTC m=+5.914181292 container cleanup ddad0f843aa7e13fbedf86f7a47536cd80b5e618814e81e6130b4f3e4bc8b418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=) 2024-09-10T18:26:01.255 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:26:01 smithi099 bash[77204]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3 2024-09-10T18:26:01.618 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:26:01 smithi099 podman[77220]: 2024-09-10 18:26:01.264627102 +0000 UTC m=+0.612678306 container remove ddad0f843aa7e13fbedf86f7a47536cd80b5e618814e81e6130b4f3e4bc8b418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T18:26:01.869 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:26:01 smithi099 podman[77305]: 2024-09-10 18:26:01.519207416 +0000 UTC m=+0.015607148 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:26:01.869 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:26:01 smithi099 podman[77305]: 2024-09-10 18:26:01.640416084 +0000 UTC m=+0.136815882 container create 0e3ac707b4ae377921abb65e824910598088f623ff7d54aa0b51ee5cfc1f059b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T18:26:01.869 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:26:01 smithi099 podman[77305]: 2024-09-10 18:26:01.867748884 +0000 UTC m=+0.364148618 container init 0e3ac707b4ae377921abb65e824910598088f623ff7d54aa0b51ee5cfc1f059b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:26:02.240 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:02 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:02.055+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:02.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:01 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:01.982+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:02.241 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:01 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:01.894+0000 7fe090cb2640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:33.112560+0000 front 2024-09-10T18:25:33.112428+0000 (oldest deadline 2024-09-10T18:25:58.412477+0000) 2024-09-10T18:26:02.255 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:26:01 smithi099 podman[77305]: 2024-09-10 18:26:01.871319276 +0000 UTC m=+0.367719008 container start 0e3ac707b4ae377921abb65e824910598088f623ff7d54aa0b51ee5cfc1f059b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T18:26:02.255 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:26:01 smithi099 podman[77305]: 2024-09-10 18:26:01.951253615 +0000 UTC m=+0.447653361 container attach 0e3ac707b4ae377921abb65e824910598088f623ff7d54aa0b51ee5cfc1f059b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:26:02.255 INFO:journalctl@ceph.osd.3.smithi099.stdout:Sep 10 18:26:02 smithi099 podman[77305]: 2024-09-10 18:26:02.041282705 +0000 UTC m=+0.537682442 container died 0e3ac707b4ae377921abb65e824910598088f623ff7d54aa0b51ee5cfc1f059b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T18:26:02.656 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T18:26:02.695 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:26:02.695 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T18:26:02.695 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T18:26:02.696 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.4 2024-09-10T18:26:02.723 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:02 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:02.291+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:02.991 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:02 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:02.936+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:02.991 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:02 smithi133 systemd[1]: Stopping Ceph osd.4 for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:26:02.991 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:02 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:02.904+0000 7fe090cb2640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:33.112560+0000 front 2024-09-10T18:25:33.112428+0000 (oldest deadline 2024-09-10T18:25:58.412477+0000) 2024-09-10T18:26:03.490 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:03.017+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:03.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:03.249+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:03.491 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:03.014+0000 7fe0976a0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:26:03.491 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:03.014+0000 7fe0976a0640 -1 osd.4 380 *** Got signal Terminated *** 2024-09-10T18:26:03.491 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:03.014+0000 7fe0976a0640 -1 osd.4 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:26:04.005 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:03 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:03.530Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi099.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.99:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi133.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.133:8443: connect: connection refused" 2024-09-10T18:26:04.005 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:03 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:03.530Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi099.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.99:8443: connect: connection refused" 2024-09-10T18:26:04.005 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:03 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:03.531Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi133.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.133:8443: connect: connection refused" 2024-09-10T18:26:04.241 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:03.868+0000 7fe090cb2640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:33.112560+0000 front 2024-09-10T18:25:33.112428+0000 (oldest deadline 2024-09-10T18:25:58.412477+0000) 2024-09-10T18:26:04.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:04 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:04.056+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:04.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:03 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:03.983+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:04.740 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:04 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:04.274+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:05.240 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:05 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:05.082+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:05.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:04 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:04.996+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:05.241 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:04 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:04.886+0000 7fe090cb2640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:33.112560+0000 front 2024-09-10T18:25:33.112428+0000 (oldest deadline 2024-09-10T18:25:58.412477+0000) 2024-09-10T18:26:05.740 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:05 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:05.243+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:06.241 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:05 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:05.922+0000 7fe090cb2640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:33.112560+0000 front 2024-09-10T18:25:33.112428+0000 (oldest deadline 2024-09-10T18:25:58.412477+0000) 2024-09-10T18:26:06.241 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:05 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:05.922+0000 7fe090cb2640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.114798+0000 front 2024-09-10T18:25:44.114830+0000 (oldest deadline 2024-09-10T18:26:05.814791+0000) 2024-09-10T18:26:06.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:06 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:06.048+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:06.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:06 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:06.048+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:43.401978+0000 front 2024-09-10T18:25:43.401884+0000 (oldest deadline 2024-09-10T18:26:05.101866+0000) 2024-09-10T18:26:06.242 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:05 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:05.983+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:06.242 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:05 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:05.983+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:06.740 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:06 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:06.274+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:07.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:07.006+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:07.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:07.006+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:43.401978+0000 front 2024-09-10T18:25:43.401884+0000 (oldest deadline 2024-09-10T18:26:05.101866+0000) 2024-09-10T18:26:07.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:06 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:06.936+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:07.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:06 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:06.936+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:07.241 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:06 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:06.953+0000 7fe090cb2640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:33.112560+0000 front 2024-09-10T18:25:33.112428+0000 (oldest deadline 2024-09-10T18:25:58.412477+0000) 2024-09-10T18:26:07.242 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:06 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:06.953+0000 7fe090cb2640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.114798+0000 front 2024-09-10T18:25:44.114830+0000 (oldest deadline 2024-09-10T18:26:05.814791+0000) 2024-09-10T18:26:07.741 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:07.309+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:07.741 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:07.309+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:08.241 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:07.952+0000 7fe090cb2640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:33.112560+0000 front 2024-09-10T18:25:33.112428+0000 (oldest deadline 2024-09-10T18:25:58.412477+0000) 2024-09-10T18:26:08.241 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4[36124]: 2024-09-10T18:26:07.952+0000 7fe090cb2640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.114798+0000 front 2024-09-10T18:25:44.114830+0000 (oldest deadline 2024-09-10T18:26:05.814791+0000) 2024-09-10T18:26:08.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:08.015+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:08.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:08.015+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:43.401978+0000 front 2024-09-10T18:25:43.401884+0000 (oldest deadline 2024-09-10T18:26:05.101866+0000) 2024-09-10T18:26:08.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:07.936+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:08.242 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:07 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:07.936+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:08.616 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:08 smithi133 podman[65672]: 2024-09-10 18:26:08.35907974 +0000 UTC m=+5.468578781 container died ec2ae3ef9f75b26092ec8f6d613c6b6d733ec63d2385895be05c0c36c74e5ef5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T18:26:08.616 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:08.355+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:08.616 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:08.355+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:08.941 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:08 smithi133 podman[65672]: 2024-09-10 18:26:08.713700662 +0000 UTC m=+5.823199692 container cleanup ec2ae3ef9f75b26092ec8f6d613c6b6d733ec63d2385895be05c0c36c74e5ef5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:26:08.941 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:08 smithi133 bash[65672]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4 2024-09-10T18:26:09.240 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:08 smithi133 podman[65687]: 2024-09-10 18:26:08.98999669 +0000 UTC m=+0.630880284 container remove ec2ae3ef9f75b26092ec8f6d613c6b6d733ec63d2385895be05c0c36c74e5ef5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2) 2024-09-10T18:26:09.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:08.974+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:09.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:08.974+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:43.401978+0000 front 2024-09-10T18:25:43.401884+0000 (oldest deadline 2024-09-10T18:26:05.101866+0000) 2024-09-10T18:26:09.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:08.939+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:09.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:08 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:08.939+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:09.594 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:09.329+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:09.594 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:09.329+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:09.595 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:09 smithi133 podman[65778]: 2024-09-10 18:26:09.238525948 +0000 UTC m=+0.016290463 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:26:09.595 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:09 smithi133 podman[65778]: 2024-09-10 18:26:09.340794636 +0000 UTC m=+0.118559186 container create 8327b02515de5c6dfa8c19f6a0acdceb8dd00da0489ad4110a33456e212703d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T18:26:09.595 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:09 smithi133 podman[65778]: 2024-09-10 18:26:09.593248305 +0000 UTC m=+0.371012820 container init 8327b02515de5c6dfa8c19f6a0acdceb8dd00da0489ad4110a33456e212703d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T18:26:09.953 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:09 smithi133 podman[65778]: 2024-09-10 18:26:09.596903117 +0000 UTC m=+0.374667623 container start 8327b02515de5c6dfa8c19f6a0acdceb8dd00da0489ad4110a33456e212703d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T18:26:09.954 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:09 smithi133 podman[65778]: 2024-09-10 18:26:09.66810992 +0000 UTC m=+0.445874434 container attach 8327b02515de5c6dfa8c19f6a0acdceb8dd00da0489ad4110a33456e212703d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T18:26:09.954 INFO:journalctl@ceph.osd.4.smithi133.stdout:Sep 10 18:26:09 smithi133 podman[65778]: 2024-09-10 18:26:09.770269123 +0000 UTC m=+0.548033630 container died 8327b02515de5c6dfa8c19f6a0acdceb8dd00da0489ad4110a33456e212703d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:26:10.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:10 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:10.006+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:10.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:10 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:10.006+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:43.401978+0000 front 2024-09-10T18:25:43.401884+0000 (oldest deadline 2024-09-10T18:26:05.101866+0000) 2024-09-10T18:26:10.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:09.951+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:10.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:09 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:09.951+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:10.376 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T18:26:10.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:26:10.454 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T18:26:10.454 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T18:26:10.455 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.5 2024-09-10T18:26:10.741 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:10 smithi133 systemd[1]: Stopping Ceph osd.5 for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:26:10.741 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:10 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:10.283+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:10.741 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:10 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:10.283+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:11.240 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:10 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:10.947+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:11.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:10 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:10.947+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:11.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:10 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:10.784+0000 7fca327fc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:26:11.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:10 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:10.784+0000 7fca327fc640 -1 osd.5 380 *** Got signal Terminated *** 2024-09-10T18:26:11.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:10 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:10.784+0000 7fca327fc640 -1 osd.5 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:26:11.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:11 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:11.018+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:11.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:11 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:11.018+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:43.401978+0000 front 2024-09-10T18:25:43.401884+0000 (oldest deadline 2024-09-10T18:26:05.101866+0000) 2024-09-10T18:26:11.741 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:11 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:11.310+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:11.741 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:11 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:11.310+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:12.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:11 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:11.912+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:12.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:11 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:11.912+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:12.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:12 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:12.059+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:12.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:12 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:12.059+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:43.401978+0000 front 2024-09-10T18:25:43.401884+0000 (oldest deadline 2024-09-10T18:26:05.101866+0000) 2024-09-10T18:26:12.740 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:12 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:12.302+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:12.741 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:12 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:12.302+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:13.240 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:13 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:13.047+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:13.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:13 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:13.047+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:43.401978+0000 front 2024-09-10T18:25:43.401884+0000 (oldest deadline 2024-09-10T18:26:05.101866+0000) 2024-09-10T18:26:13.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:12 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:12.933+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:13.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:12 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:12.933+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:13.740 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:13 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:13.260+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:13.741 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:13 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:13.260+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:13.916 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:13 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:13.530Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi133.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.133:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi099.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.99:8443: connect: connection refused" 2024-09-10T18:26:13.916 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:13 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:13.531Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi099.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.99:8443: connect: connection refused" 2024-09-10T18:26:13.916 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:13 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:13.532Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi133.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.133:8443: connect: connection refused" 2024-09-10T18:26:14.240 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:13 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:13.893+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:14.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:13 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:13.893+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:14.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:14 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:14.011+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:14.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:14 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:14.011+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:43.401978+0000 front 2024-09-10T18:25:43.401884+0000 (oldest deadline 2024-09-10T18:26:05.101866+0000) 2024-09-10T18:26:14.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:14 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:14.011+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:50.403041+0000 front 2024-09-10T18:25:50.403031+0000 (oldest deadline 2024-09-10T18:26:13.902904+0000) 2024-09-10T18:26:14.740 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:14 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:14.249+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:14.741 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:14 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:14.249+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:15.240 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:14 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:14.864+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:15.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:14 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:14.864+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:15.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:14 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:14.966+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.800819+0000 front 2024-09-10T18:25:35.800860+0000 (oldest deadline 2024-09-10T18:25:58.100742+0000) 2024-09-10T18:26:15.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:14 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:14.966+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:43.401978+0000 front 2024-09-10T18:25:43.401884+0000 (oldest deadline 2024-09-10T18:26:05.101866+0000) 2024-09-10T18:26:15.241 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:14 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5[41904]: 2024-09-10T18:26:14.966+0000 7fca2d611640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:50.403041+0000 front 2024-09-10T18:25:50.403031+0000 (oldest deadline 2024-09-10T18:26:13.902904+0000) 2024-09-10T18:26:15.740 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:15 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:15.283+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:15.741 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:15 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:15.284+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:16.188 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:15 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:15.865+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:16.188 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:15 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:15.865+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:16.491 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:16 smithi133 podman[65879]: 2024-09-10 18:26:16.186744499 +0000 UTC m=+5.514863125 container died cbfeb512c8fcba9a1403756c7b4e40ee9b7c79156883214809bb756764c97607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T18:26:16.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:16.264+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:16.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:16.264+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:16.809 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:16 smithi133 podman[65879]: 2024-09-10 18:26:16.573756349 +0000 UTC m=+5.901874975 container cleanup cbfeb512c8fcba9a1403756c7b4e40ee9b7c79156883214809bb756764c97607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=) 2024-09-10T18:26:16.810 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:16 smithi133 bash[65879]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5 2024-09-10T18:26:17.117 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:16 smithi133 podman[65895]: 2024-09-10 18:26:16.808483115 +0000 UTC m=+0.620795656 container remove cbfeb512c8fcba9a1403756c7b4e40ee9b7c79156883214809bb756764c97607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T18:26:17.118 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:16.876+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:17.118 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:16 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:16.876+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:17.371 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:17 smithi133 podman[65986]: 2024-09-10 18:26:17.019023781 +0000 UTC m=+0.016070262 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:26:17.371 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:17 smithi133 podman[65986]: 2024-09-10 18:26:17.134319232 +0000 UTC m=+0.131365756 container create 4450438662a174c97ad43484e555f920c165fb45d208fb5f5db1ce90e2bce6de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T18:26:17.371 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:17 smithi133 podman[65986]: 2024-09-10 18:26:17.369878766 +0000 UTC m=+0.366925250 container init 4450438662a174c97ad43484e555f920c165fb45d208fb5f5db1ce90e2bce6de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T18:26:17.371 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:17.256+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:17.371 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:17.256+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:17.372 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:17.256+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:17.671 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:17 smithi133 podman[65986]: 2024-09-10 18:26:17.373146511 +0000 UTC m=+0.370192990 container start 4450438662a174c97ad43484e555f920c165fb45d208fb5f5db1ce90e2bce6de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS) 2024-09-10T18:26:17.671 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:17 smithi133 podman[65986]: 2024-09-10 18:26:17.669970908 +0000 UTC m=+0.667017399 container attach 4450438662a174c97ad43484e555f920c165fb45d208fb5f5db1ce90e2bce6de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:26:17.671 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 18:26:17 smithi133 podman[65986]: 2024-09-10 18:26:17.670228517 +0000 UTC m=+0.667274997 container died 4450438662a174c97ad43484e555f920c165fb45d208fb5f5db1ce90e2bce6de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T18:26:17.991 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:17.835+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:17.991 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:17.835+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:17.991 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:17 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:17.835+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:52.089990+0000 front 2024-09-10T18:25:52.089909+0000 (oldest deadline 2024-09-10T18:26:17.389856+0000) 2024-09-10T18:26:18.333 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:18.237+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:18.333 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:18.237+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:18.333 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:18.237+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:18.362 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T18:26:18.403 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:26:18.403 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T18:26:18.404 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T18:26:18.404 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.6 2024-09-10T18:26:18.711 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:18 smithi133 systemd[1]: Stopping Ceph osd.6 for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:26:18.990 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:18.709+0000 7f48e0cad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:26:18.991 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:18.709+0000 7f48e0cad640 -1 osd.6 380 *** Got signal Terminated *** 2024-09-10T18:26:18.991 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:18.709+0000 7f48e0cad640 -1 osd.6 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:26:18.991 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:18.861+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:18.991 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:18.861+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:18.991 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:18 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:18.861+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:52.089990+0000 front 2024-09-10T18:25:52.089909+0000 (oldest deadline 2024-09-10T18:26:17.389856+0000) 2024-09-10T18:26:19.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:19 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:19.198+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:19.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:19 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:19.198+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:19.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:19 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:19.198+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:19.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:19 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:19.198+0000 7fc45032a640 -1 osd.7 380 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24463.0:465 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e380) 2024-09-10T18:26:20.240 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:19 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:19.910+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:20.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:19 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:19.910+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:20.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:19 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:19.910+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:52.089990+0000 front 2024-09-10T18:25:52.089909+0000 (oldest deadline 2024-09-10T18:26:17.389856+0000) 2024-09-10T18:26:20.241 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:20.158+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:20.241 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:20.158+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:20.241 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:20.158+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:20.241 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:20.158+0000 7fc45032a640 -1 osd.7 380 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24463.0:465 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e380) 2024-09-10T18:26:21.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:20.908+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:21.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:20.908+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:21.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:20.908+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:52.089990+0000 front 2024-09-10T18:25:52.089909+0000 (oldest deadline 2024-09-10T18:26:17.389856+0000) 2024-09-10T18:26:21.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:20 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:20.908+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6815 osd.3 since back 2024-09-10T18:25:59.690727+0000 front 2024-09-10T18:25:59.690777+0000 (oldest deadline 2024-09-10T18:26:20.790648+0000) 2024-09-10T18:26:21.241 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:21.109+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:21.242 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:21.109+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:21.242 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:21.109+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:21.242 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:21.109+0000 7fc45032a640 -1 osd.7 380 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24463.0:465 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e380) 2024-09-10T18:26:22.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:21.920+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:22.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:21.920+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:22.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:21.920+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:52.089990+0000 front 2024-09-10T18:25:52.089909+0000 (oldest deadline 2024-09-10T18:26:17.389856+0000) 2024-09-10T18:26:22.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:21 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:21.920+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6815 osd.3 since back 2024-09-10T18:25:59.690727+0000 front 2024-09-10T18:25:59.690777+0000 (oldest deadline 2024-09-10T18:26:20.790648+0000) 2024-09-10T18:26:22.241 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:22.108+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:22.242 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:22.108+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:22.242 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:22.108+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:22.242 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:22.108+0000 7fc45032a640 -1 osd.7 380 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24463.0:465 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e380) 2024-09-10T18:26:23.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:22.954+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:35.787719+0000 front 2024-09-10T18:25:35.787643+0000 (oldest deadline 2024-09-10T18:25:59.887525+0000) 2024-09-10T18:26:23.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:22.954+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:41.588435+0000 front 2024-09-10T18:25:41.588414+0000 (oldest deadline 2024-09-10T18:26:05.688376+0000) 2024-09-10T18:26:23.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:22.954+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:52.089990+0000 front 2024-09-10T18:25:52.089909+0000 (oldest deadline 2024-09-10T18:26:17.389856+0000) 2024-09-10T18:26:23.241 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:22 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6[47720]: 2024-09-10T18:26:22.954+0000 7f48dcac4640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.99:6815 osd.3 since back 2024-09-10T18:25:59.690727+0000 front 2024-09-10T18:25:59.690777+0000 (oldest deadline 2024-09-10T18:26:20.790648+0000) 2024-09-10T18:26:23.241 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:23.109+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:23.241 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:23.109+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:23.241 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:23.109+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:23.242 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:23.109+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6815 osd.3 since back 2024-09-10T18:25:57.194053+0000 front 2024-09-10T18:25:57.194012+0000 (oldest deadline 2024-09-10T18:26:23.093977+0000) 2024-09-10T18:26:23.242 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:23 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:23.109+0000 7fc45032a640 -1 osd.7 380 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24463.0:465 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e380) 2024-09-10T18:26:24.006 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:23.531Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi099.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.99:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi133.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.133:8443: connect: connection refused" 2024-09-10T18:26:24.006 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:23.532Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi099.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.99:8443: connect: connection refused" 2024-09-10T18:26:24.006 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:23 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:23.532Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi133.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.133:8443: connect: connection refused" 2024-09-10T18:26:24.486 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:24 smithi133 podman[66089]: 2024-09-10 18:26:24.213523169 +0000 UTC m=+5.595466447 container died 83d4f3b704811bd9b6c09e68c1b0467449988342789fe13ce510aac93e08d099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T18:26:24.487 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:24 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:24.084+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:24.487 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:24 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:24.084+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:24.487 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:24 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:24.084+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:24.487 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:24 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:24.084+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6815 osd.3 since back 2024-09-10T18:25:57.194053+0000 front 2024-09-10T18:25:57.194012+0000 (oldest deadline 2024-09-10T18:26:23.093977+0000) 2024-09-10T18:26:24.487 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:24 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:24.084+0000 7fc45032a640 -1 osd.7 380 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24463.0:465 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e380) 2024-09-10T18:26:24.740 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:24 smithi133 podman[66089]: 2024-09-10 18:26:24.575607654 +0000 UTC m=+5.957550995 container cleanup 83d4f3b704811bd9b6c09e68c1b0467449988342789fe13ce510aac93e08d099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T18:26:24.741 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:24 smithi133 bash[66089]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6 2024-09-10T18:26:25.194 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:24 smithi133 podman[66107]: 2024-09-10 18:26:24.868407268 +0000 UTC m=+0.647376370 container remove 83d4f3b704811bd9b6c09e68c1b0467449988342789fe13ce510aac93e08d099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T18:26:25.194 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:25 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:25.072+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:25.195 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:25 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:25.072+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:25.195 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:25 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:25.072+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:25.195 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:25 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:25.072+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6815 osd.3 since back 2024-09-10T18:25:57.194053+0000 front 2024-09-10T18:25:57.194012+0000 (oldest deadline 2024-09-10T18:26:23.093977+0000) 2024-09-10T18:26:25.195 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:25 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:25.072+0000 7fc45032a640 -1 osd.7 380 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24463.0:465 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e380) 2024-09-10T18:26:25.449 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:25 smithi133 podman[66192]: 2024-09-10 18:26:25.095586639 +0000 UTC m=+0.015480936 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:26:25.449 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:25 smithi133 podman[66192]: 2024-09-10 18:26:25.227814532 +0000 UTC m=+0.147708887 container create 8f52de9678023da1bcd8486c938a27dc24ef49011f2728371f2f9ed9ecf2dd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T18:26:25.449 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:25 smithi133 podman[66192]: 2024-09-10 18:26:25.447094234 +0000 UTC m=+0.366988532 container init 8f52de9678023da1bcd8486c938a27dc24ef49011f2728371f2f9ed9ecf2dd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:26:25.741 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:25 smithi133 podman[66192]: 2024-09-10 18:26:25.450597716 +0000 UTC m=+0.370492019 container start 8f52de9678023da1bcd8486c938a27dc24ef49011f2728371f2f9ed9ecf2dd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T18:26:25.741 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:25 smithi133 podman[66192]: 2024-09-10 18:26:25.554946248 +0000 UTC m=+0.474840548 container attach 8f52de9678023da1bcd8486c938a27dc24ef49011f2728371f2f9ed9ecf2dd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:26:25.741 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:25 smithi133 podman[66192]: 2024-09-10 18:26:25.621757294 +0000 UTC m=+0.541651583 container died 8f52de9678023da1bcd8486c938a27dc24ef49011f2728371f2f9ed9ecf2dd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD) 2024-09-10T18:26:26.169 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T18:26:26.193 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:26 smithi133 podman[66215]: 2024-09-10 18:26:26.141620717 +0000 UTC m=+0.509740707 container remove 8f52de9678023da1bcd8486c938a27dc24ef49011f2728371f2f9ed9ecf2dd48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T18:26:26.193 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:26 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.6.service: Deactivated successfully. 2024-09-10T18:26:26.194 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:26 smithi133 systemd[1]: Stopped Ceph osd.6 for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:26:26.194 INFO:journalctl@ceph.osd.6.smithi133.stdout:Sep 10 18:26:26 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.6.service: Consumed 12.060s CPU time. 2024-09-10T18:26:26.194 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:26 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:26.049+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:26.194 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:26 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:26.049+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:26.194 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:26 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:26.049+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:26.194 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:26 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:26.049+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6815 osd.3 since back 2024-09-10T18:25:57.194053+0000 front 2024-09-10T18:25:57.194012+0000 (oldest deadline 2024-09-10T18:26:23.093977+0000) 2024-09-10T18:26:26.194 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:26 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:26.049+0000 7fc45032a640 -1 osd.7 380 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24463.0:465 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e380) 2024-09-10T18:26:26.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:26:26.210 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T18:26:26.210 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T18:26:26.210 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.7 2024-09-10T18:26:26.490 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:26 smithi133 systemd[1]: Stopping Ceph osd.7 for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:26:26.991 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:26 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:26.543+0000 7fc456d18640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:26:26.991 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:26 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:26.543+0000 7fc456d18640 -1 osd.7 380 *** Got signal Terminated *** 2024-09-10T18:26:26.991 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:26 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:26.543+0000 7fc456d18640 -1 osd.7 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:26:27.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:27.063+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:27.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:27.063+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:27.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:27.063+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:27.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:27.063+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6815 osd.3 since back 2024-09-10T18:25:57.194053+0000 front 2024-09-10T18:25:57.194012+0000 (oldest deadline 2024-09-10T18:26:23.093977+0000) 2024-09-10T18:26:27.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:27 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:27.063+0000 7fc45032a640 -1 osd.7 380 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24463.0:465 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e380) 2024-09-10T18:26:28.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:28 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:28.059+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:28.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:28 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:28.059+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:28.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:28 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:28.059+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:28.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:28 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:28.059+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6815 osd.3 since back 2024-09-10T18:25:57.194053+0000 front 2024-09-10T18:25:57.194012+0000 (oldest deadline 2024-09-10T18:26:23.093977+0000) 2024-09-10T18:26:28.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:28 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:28.059+0000 7fc45032a640 -1 osd.7 380 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24463.0:465 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e380) 2024-09-10T18:26:29.392 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:29 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:29.099+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:29.392 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:29 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:29.099+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:29.392 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:29 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:29.099+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:29.392 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:29 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:29.099+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6815 osd.3 since back 2024-09-10T18:25:57.194053+0000 front 2024-09-10T18:25:57.194012+0000 (oldest deadline 2024-09-10T18:26:23.093977+0000) 2024-09-10T18:26:29.393 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:29 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:29.099+0000 7fc45032a640 -1 osd.7 380 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24463.0:465 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e380) 2024-09-10T18:26:30.408 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:30 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:30.124+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:30.409 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:30 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:30.124+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:30.409 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:30 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:30.124+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:30.409 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:30 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:30.124+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6815 osd.3 since back 2024-09-10T18:25:57.194053+0000 front 2024-09-10T18:25:57.194012+0000 (oldest deadline 2024-09-10T18:26:23.093977+0000) 2024-09-10T18:26:30.409 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:30 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:30.124+0000 7fc45032a640 -1 osd.7 380 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24463.0:465 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e380) 2024-09-10T18:26:31.490 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:31 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:31.095+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6803 osd.0 since back 2024-09-10T18:25:31.289527+0000 front 2024-09-10T18:25:31.289488+0000 (oldest deadline 2024-09-10T18:26:01.289837+0000) 2024-09-10T18:26:31.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:31 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:31.095+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6807 osd.1 since back 2024-09-10T18:25:44.190689+0000 front 2024-09-10T18:25:44.190540+0000 (oldest deadline 2024-09-10T18:26:07.090564+0000) 2024-09-10T18:26:31.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:31 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:31.095+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6811 osd.2 since back 2024-09-10T18:25:51.893594+0000 front 2024-09-10T18:25:51.893557+0000 (oldest deadline 2024-09-10T18:26:17.193506+0000) 2024-09-10T18:26:31.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:31 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:31.095+0000 7fc45032a640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.99:6815 osd.3 since back 2024-09-10T18:25:57.194053+0000 front 2024-09-10T18:25:57.194012+0000 (oldest deadline 2024-09-10T18:26:23.093977+0000) 2024-09-10T18:26:31.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:31 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7[53568]: 2024-09-10T18:26:31.095+0000 7fc45032a640 -1 osd.7 380 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24463.0:465 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e380) 2024-09-10T18:26:32.180 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:31 smithi133 podman[66295]: 2024-09-10 18:26:31.911858421 +0000 UTC m=+5.485773719 container died d0f7c0b7c2d26d8aafa235b8d6d064e0c42bbdf86d6ba2d1e78b8a0378dbffa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T18:26:32.490 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:32 smithi133 podman[66295]: 2024-09-10 18:26:32.252112716 +0000 UTC m=+5.826028020 container cleanup d0f7c0b7c2d26d8aafa235b8d6d064e0c42bbdf86d6ba2d1e78b8a0378dbffa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T18:26:32.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:32 smithi133 bash[66295]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7 2024-09-10T18:26:32.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:32 smithi133 podman[66322]: 2024-09-10 18:26:32.428069365 +0000 UTC m=+0.513305340 container remove d0f7c0b7c2d26d8aafa235b8d6d064e0c42bbdf86d6ba2d1e78b8a0378dbffa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T18:26:33.165 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:32 smithi133 podman[66412]: 2024-09-10 18:26:32.766694825 +0000 UTC m=+0.015790596 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:26:33.165 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:32 smithi133 podman[66412]: 2024-09-10 18:26:32.870248137 +0000 UTC m=+0.119343894 container create bba5bc030b44db1549aa641957cd62890aa0005b2bc28408bf7d27c34a252750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T18:26:33.165 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:33 smithi133 podman[66412]: 2024-09-10 18:26:33.080828406 +0000 UTC m=+0.329924169 container init bba5bc030b44db1549aa641957cd62890aa0005b2bc28408bf7d27c34a252750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS) 2024-09-10T18:26:33.165 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:33 smithi133 podman[66412]: 2024-09-10 18:26:33.084105921 +0000 UTC m=+0.333201696 container start bba5bc030b44db1549aa641957cd62890aa0005b2bc28408bf7d27c34a252750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T18:26:33.165 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:33 smithi133 podman[66412]: 2024-09-10 18:26:33.164075256 +0000 UTC m=+0.413171022 container attach bba5bc030b44db1549aa641957cd62890aa0005b2bc28408bf7d27c34a252750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T18:26:33.491 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:33 smithi133 podman[66412]: 2024-09-10 18:26:33.261062128 +0000 UTC m=+0.510157897 container died bba5bc030b44db1549aa641957cd62890aa0005b2bc28408bf7d27c34a252750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T18:26:33.817 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T18:26:33.841 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:33 smithi133 podman[66435]: 2024-09-10 18:26:33.792033094 +0000 UTC m=+0.520859372 container remove bba5bc030b44db1549aa641957cd62890aa0005b2bc28408bf7d27c34a252750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:26:33.841 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:33 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.7.service: Deactivated successfully. 2024-09-10T18:26:33.842 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:33 smithi133 systemd[1]: Stopped Ceph osd.7 for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:26:33.842 INFO:journalctl@ceph.osd.7.smithi133.stdout:Sep 10 18:26:33 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@osd.7.service: Consumed 13.271s CPU time. 2024-09-10T18:26:33.858 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:26:33.858 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T18:26:33.858 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-10T18:26:33.858 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-10T18:26:33.886 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:33.532Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi099.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.99:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi133.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.133:8443: connect: connection refused" 2024-09-10T18:26:33.886 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:33.533Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi099.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.99:8443: connect: connection refused" 2024-09-10T18:26:33.886 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:33 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:33.533Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi133.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.133:8443: connect: connection refused" 2024-09-10T18:26:34.154 INFO:journalctl@ceph.rgw.foo.a.smithi099.stdout:Sep 10 18:26:33 smithi099 systemd[1]: Stopping Ceph rgw.foo.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:26:34.505 INFO:journalctl@ceph.rgw.foo.a.smithi099.stdout:Sep 10 18:26:34 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-rgw-foo-a[64631]: 2024-09-10T18:26:34.152+0000 7f22d5848640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:26:34.505 INFO:journalctl@ceph.rgw.foo.a.smithi099.stdout:Sep 10 18:26:34 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-rgw-foo-a[64631]: 2024-09-10T18:26:34.152+0000 7f22d9d507c0 -1 shutting down 2024-09-10T18:26:44.005 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:43 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:43.533Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi099.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.99:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi133.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.133:8443: connect: connection refused" 2024-09-10T18:26:44.006 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:43 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:43.534Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi099.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.99:8443: connect: connection refused" 2024-09-10T18:26:44.006 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:43 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:43.534Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi133.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.133:8443: connect: connection refused" 2024-09-10T18:26:44.444 INFO:journalctl@ceph.rgw.foo.a.smithi099.stdout:Sep 10 18:26:44 smithi099 bash[77418]: time="2024-09-10T18:26:44Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-10T18:26:44.444 INFO:journalctl@ceph.rgw.foo.a.smithi099.stdout:Sep 10 18:26:44 smithi099 podman[77418]: 2024-09-10 18:26:44.174023289 +0000 UTC m=+10.125281825 container died 28bdc0b2c33d9d9fd01ce7927cdb9c3780355beeb380b24852fee49630d8fa7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-rgw-foo-a, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:26:44.755 INFO:journalctl@ceph.rgw.foo.a.smithi099.stdout:Sep 10 18:26:44 smithi099 podman[77418]: 2024-09-10 18:26:44.5503176 +0000 UTC m=+10.501576136 container cleanup 28bdc0b2c33d9d9fd01ce7927cdb9c3780355beeb380b24852fee49630d8fa7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:26:44.755 INFO:journalctl@ceph.rgw.foo.a.smithi099.stdout:Sep 10 18:26:44 smithi099 bash[77418]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-10T18:26:44.978 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-10T18:26:45.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:26:45.053 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-10T18:26:45.054 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-10T18:26:45.054 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@prometheus.a 2024-09-10T18:26:45.345 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 systemd[1]: Stopping Ceph prometheus.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:26:45.629 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:26:45.344Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:26:45.629 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:26:45.344Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T18:26:45.629 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:26:45.344Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T18:26:45.629 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:26:45.344Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T18:26:45.629 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:26:45.344Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T18:26:45.629 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:26:45.344Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T18:26:45.629 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:26:45.344Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T18:26:45.629 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:26:45.344Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T18:26:45.629 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:26:45.344Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T18:26:45.629 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:26:45.344Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T18:26:45.630 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:26:45.344Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T18:26:45.630 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a[63436]: ts=2024-09-10T18:26:45.345Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T18:26:45.630 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 podman[66516]: 2024-09-10 18:26:45.354370185 +0000 UTC m=+0.098430102 container died 11a63b3b5e9ff44cc373221290a3efbca1a6ec8c792e97510f2c08c1cb0d378c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:26:45.979 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 podman[66516]: 2024-09-10 18:26:45.743338488 +0000 UTC m=+0.487398405 container cleanup 11a63b3b5e9ff44cc373221290a3efbca1a6ec8c792e97510f2c08c1cb0d378c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:26:45.979 INFO:journalctl@ceph.prometheus.a.smithi133.stdout:Sep 10 18:26:45 smithi133 bash[66516]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T18:26:46.116 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-10T18:26:46.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:26:46.151 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-10T18:26:46.152 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T18:26:48.005 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:47 smithi099 systemd[1]: Stopping Ceph alertmanager.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:26:48.357 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:48 smithi099 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a[72635]: ts=2024-09-10T18:26:48.039Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:26:48.357 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:48 smithi099 podman[77859]: 2024-09-10 18:26:48.079729405 +0000 UTC m=+0.143585399 container died 96422768b205d051fe741146ab884a674e0509bc5f3255f29e6375be0e101e86 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:26:48.700 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:48 smithi099 podman[77859]: 2024-09-10 18:26:48.447376108 +0000 UTC m=+0.511232100 container cleanup 96422768b205d051fe741146ab884a674e0509bc5f3255f29e6375be0e101e86 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:26:48.700 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:48 smithi099 bash[77859]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:26:49.005 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:48 smithi099 podman[77871]: 2024-09-10 18:26:48.698773824 +0000 UTC m=+0.612779788 container remove 96422768b205d051fe741146ab884a674e0509bc5f3255f29e6375be0e101e86 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:26:49.005 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:48 smithi099 podman[77871]: 2024-09-10 18:26:48.791096688 +0000 UTC m=+0.705102654 volume remove fb5c9cc23e7d3bb8d2f3398e397540f58b14af42212250549d437a51d4bef438 2024-09-10T18:26:49.005 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:48 smithi099 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T18:26:49.005 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:48 smithi099 systemd[1]: Stopped Ceph alertmanager.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:26:49.005 INFO:journalctl@ceph.alertmanager.a.smithi099.stdout:Sep 10 18:26:48 smithi099 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.142s CPU time. 2024-09-10T18:26:49.355 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:26:49 smithi099 systemd[1]: Stopping Ceph node-exporter.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:26:49.686 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:26:49 smithi099 podman[77977]: 2024-09-10 18:26:49.354121672 +0000 UTC m=+0.126098415 container died 570576c412f701f4ffbb7ea57117b7ca7852756ea2050eb8d7148fb749d52622 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:26:49.686 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:26:49 smithi099 podman[77977]: 2024-09-10 18:26:49.685141936 +0000 UTC m=+0.457118685 container cleanup 570576c412f701f4ffbb7ea57117b7ca7852756ea2050eb8d7148fb749d52622 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:26:50.005 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:26:49 smithi099 bash[77977]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a 2024-09-10T18:26:50.005 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:26:49 smithi099 podman[77988]: 2024-09-10 18:26:49.844832678 +0000 UTC m=+0.494624007 container remove 570576c412f701f4ffbb7ea57117b7ca7852756ea2050eb8d7148fb749d52622 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:26:50.005 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:26:49 smithi099 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:26:50.471 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:26:50 smithi099 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-10T18:26:50.471 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:26:50 smithi099 systemd[1]: Stopped Ceph node-exporter.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:26:50.471 INFO:journalctl@ceph.node-exporter.a.smithi099.stdout:Sep 10 18:26:50 smithi099 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 2.334s CPU time. 2024-09-10T18:26:50.687 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T18:26:52.491 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:26:52 smithi133 systemd[1]: Stopping Ceph iscsi.iscsi.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:26:52.491 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:26:52 smithi133 ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a[58556]: debug Shutdown received 2024-09-10T18:27:02.531 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:02 smithi133 bash[66888]: time="2024-09-10T18:27:02Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-10T18:27:02.532 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:02 smithi133 podman[66888]: 2024-09-10 18:27:02.269822833 +0000 UTC m=+10.115933084 container died 64ebfc7716f38f63ec30bbbe956ce9d2147bfa86b05b3b18867ff501714178f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:27:02.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:02 smithi133 podman[66888]: 2024-09-10 18:27:02.629219451 +0000 UTC m=+10.475329701 container cleanup 64ebfc7716f38f63ec30bbbe956ce9d2147bfa86b05b3b18867ff501714178f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:27:02.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:02 smithi133 bash[66888]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-10T18:27:03.228 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:02 smithi133 podman[66903]: 2024-09-10 18:27:02.85543533 +0000 UTC m=+0.581547532 container remove 64ebfc7716f38f63ec30bbbe956ce9d2147bfa86b05b3b18867ff501714178f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T18:27:03.228 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:02 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-10T18:27:03.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:03 smithi133 podman[66954]: 2024-09-10 18:27:03.227172521 +0000 UTC m=+0.119032127 container died aadb7b707e640754224abf13181ac3b230df9abb43e1d651931bb75d0d84abe8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T18:27:03.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:03 smithi133 podman[66954]: 2024-09-10 18:27:03.582862424 +0000 UTC m=+0.474722028 container cleanup aadb7b707e640754224abf13181ac3b230df9abb43e1d651931bb75d0d84abe8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:27:03.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:03 smithi133 bash[66954]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-10T18:27:03.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:03 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-10T18:27:03.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:03 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 58472 (conmon) remains running after unit stopped. 2024-09-10T18:27:03.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:03 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 66967 (podman) remains running after unit stopped. 2024-09-10T18:27:03.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:03 smithi133 systemd[1]: Stopped Ceph iscsi.iscsi.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:27:03.887 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:03 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 2.484s CPU time. 2024-09-10T18:27:04.225 INFO:journalctl@ceph.iscsi.iscsi.a.smithi133.stdout:Sep 10 18:27:03 smithi133 podman[66967]: 2024-09-10 18:27:03.884760912 +0000 UTC m=+0.654056419 container remove aadb7b707e640754224abf13181ac3b230df9abb43e1d651931bb75d0d84abe8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T18:27:04.225 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:27:03 smithi133 systemd[1]: Stopping Ceph grafana.a for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:27:04.490 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:27:04 smithi133 podman[67090]: 2024-09-10 18:27:04.223328926 +0000 UTC m=+0.090961353 container died 159ca1ecb4a7e339c2ebbe520fd13887b558a9f105ae7a98352dd8e043e2a0cd (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, version=8.5, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.tags=base rhel8, vendor=Red Hat, Inc., description=Ceph Grafana Container, io.openshift.expose-services=, maintainer=Paul Cuzner , com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, build-date=2022-04-29T11:48:11.055897, distribution-scope=public, io.buildah.version=1.32.0, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration) 2024-09-10T18:27:04.491 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:27:04 smithi133 podman[67090]: 2024-09-10 18:27:04.470662888 +0000 UTC m=+0.338295305 container cleanup 159ca1ecb4a7e339c2ebbe520fd13887b558a9f105ae7a98352dd8e043e2a0cd (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, io.openshift.tags=base rhel8, vcs-type=git, io.openshift.expose-services=, io.buildah.version=1.32.0, com.redhat.component=ubi8-container, version=8.5, distribution-scope=public, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, name=ubi8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-09-10T18:27:04.491 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:27:04 smithi133 bash[67090]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a 2024-09-10T18:27:05.082 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:27:04 smithi133 podman[67102]: 2024-09-10 18:27:04.74651896 +0000 UTC m=+0.521937040 container remove 159ca1ecb4a7e339c2ebbe520fd13887b558a9f105ae7a98352dd8e043e2a0cd (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-grafana-a, distribution-scope=public, vendor=Red Hat, Inc., maintainer=Paul Cuzner , io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=Ceph Grafana Container, io.openshift.tags=base rhel8, name=ubi8, architecture=x86_64, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, com.redhat.component=ubi8-container, version=8.5, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0, build-date=2022-04-29T11:48:11.055897, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com) 2024-09-10T18:27:05.082 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:27:04 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:27:05.082 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:27:04 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-10T18:27:05.082 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:27:04 smithi133 systemd[1]: Stopped Ceph grafana.a for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:27:05.082 INFO:journalctl@ceph.grafana.a.smithi133.stdout:Sep 10 18:27:04 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@grafana.a.service: Consumed 6.074s CPU time. 2024-09-10T18:27:05.490 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:27:05 smithi133 systemd[1]: Stopping Ceph node-exporter.b for e9807d90-6f9f-11ef-bcea-c7b262605968... 2024-09-10T18:27:05.490 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:27:05 smithi133 podman[67212]: 2024-09-10 18:27:05.318137074 +0000 UTC m=+0.101377256 container died 09d061edb509245913129ac8e697d9b1d918eccf030657153a422d28ac34300c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:27:05.826 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:27:05 smithi133 podman[67212]: 2024-09-10 18:27:05.623763725 +0000 UTC m=+0.407003903 container cleanup 09d061edb509245913129ac8e697d9b1d918eccf030657153a422d28ac34300c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:27:05.826 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:27:05 smithi133 bash[67212]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b 2024-09-10T18:27:06.223 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:27:05 smithi133 podman[67227]: 2024-09-10 18:27:05.824865084 +0000 UTC m=+0.510425297 container remove 09d061edb509245913129ac8e697d9b1d918eccf030657153a422d28ac34300c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-e9807d90-6f9f-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:27:06.223 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:27:05 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:27:06.223 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:27:06 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-10T18:27:06.223 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:27:06 smithi133 systemd[1]: Stopped Ceph node-exporter.b for e9807d90-6f9f-11ef-bcea-c7b262605968. 2024-09-10T18:27:06.223 INFO:journalctl@ceph.node-exporter.b.smithi133.stdout:Sep 10 18:27:06 smithi133 systemd[1]: ceph-e9807d90-6f9f-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 2.376s CPU time. 2024-09-10T18:27:06.682 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:27:06.716 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:27:06.743 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T18:27:06.745 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900198/remote/smithi099/crash 2024-09-10T18:27:06.746 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/crash -- . 2024-09-10T18:27:06.796 INFO:teuthology.orchestra.run.smithi099.stderr:tar: /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T18:27:06.796 INFO:teuthology.orchestra.run.smithi099.stderr:tar: Error is not recoverable: exiting now 2024-09-10T18:27:06.798 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900198/remote/smithi133/crash 2024-09-10T18:27:06.799 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/crash -- . 2024-09-10T18:27:06.843 INFO:teuthology.orchestra.run.smithi133.stderr:tar: /var/lib/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T18:27:06.843 INFO:teuthology.orchestra.run.smithi133.stderr:tar: Error is not recoverable: exiting now 2024-09-10T18:27:06.844 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T18:27:06.844 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v '\(OBJECT_' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-10T18:27:06.874 INFO:teuthology.orchestra.run.smithi099.stderr:grep: /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-10T18:27:06.876 INFO:tasks.cephadm:Compressing logs... 2024-09-10T18:27:06.876 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T18:27:06.919 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T18:27:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-10T18:27:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:: No such file or directory 2024-09-10T18:27:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T18:27:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T18:27:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/tcmu-runner.log: 63.5% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-10T18:27:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T18:27:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T18:27:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T18:27:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T18:27:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T18:27:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T18:27:06.949 INFO:teuthology.orchestra.run.smithi099.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T18:27:06.950 INFO:teuthology.orchestra.run.smithi099.stderr:: No such file or directory 2024-09-10T18:27:06.950 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T18:27:06.950 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T18:27:06.950 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T18:27:06.950 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T18:27:06.950 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T18:27:06.950 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T18:27:06.950 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-10T18:27:06.950 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T18:27:06.950 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: 58.5% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-10T18:27:06.950 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T18:27:06.951 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T18:27:06.953 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T18:27:06.955 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T18:27:06.957 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.3.log: 91.2% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T18:27:06.967 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T18:27:07.136 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T18:27:07.278 INFO:teuthology.orchestra.run.smithi099.stderr: 91.6% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T18:27:07.315 INFO:teuthology.orchestra.run.smithi133.stderr: 91.7% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T18:27:07.691 INFO:teuthology.orchestra.run.smithi099.stderr: 91.0% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T18:27:08.671 INFO:teuthology.orchestra.run.smithi099.stderr: 94.1% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T18:27:08.797 INFO:teuthology.orchestra.run.smithi133.stderr: 94.0% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T18:27:08.842 INFO:teuthology.orchestra.run.smithi133.stderr: 94.1% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T18:27:08.960 INFO:teuthology.orchestra.run.smithi133.stderr: 94.0% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T18:27:09.003 INFO:teuthology.orchestra.run.smithi133.stderr: 94.2% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T18:27:09.005 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-09-10T18:27:09.005 INFO:teuthology.orchestra.run.smithi133.stderr:real 0m2.071s 2024-09-10T18:27:09.005 INFO:teuthology.orchestra.run.smithi133.stderr:user 0m7.900s 2024-09-10T18:27:09.005 INFO:teuthology.orchestra.run.smithi133.stderr:sys 0m0.323s 2024-09-10T18:27:09.009 INFO:teuthology.orchestra.run.smithi099.stderr: 94.2% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T18:27:09.189 INFO:teuthology.orchestra.run.smithi099.stderr: 94.1% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T18:27:09.417 INFO:teuthology.orchestra.run.smithi099.stderr: 94.2% -- replaced with /var/log/ceph/e9807d90-6f9f-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T18:27:09.419 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-10T18:27:09.419 INFO:teuthology.orchestra.run.smithi099.stderr:real 0m2.486s 2024-09-10T18:27:09.419 INFO:teuthology.orchestra.run.smithi099.stderr:user 0m9.480s 2024-09-10T18:27:09.419 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m0.317s 2024-09-10T18:27:09.419 INFO:tasks.cephadm:Archiving logs... 2024-09-10T18:27:09.419 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900198/remote/smithi099/log 2024-09-10T18:27:09.420 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T18:27:10.764 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900198/remote/smithi133/log 2024-09-10T18:27:10.765 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T18:27:11.901 INFO:tasks.cephadm:Removing cluster... 2024-09-10T18:27:11.901 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 --force 2024-09-10T18:27:12.472 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e9807d90-6f9f-11ef-bcea-c7b262605968 --force 2024-09-10T18:27:12.959 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T18:27:12.959 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T18:27:12.975 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T18:27:12.991 INFO:tasks.cephadm:Teardown complete 2024-09-10T18:27:12.991 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T18:27:13.001 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T18:27:13.001 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T18:27:13.018 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T18:27:13.095 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T18:27:13.104 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T18:27:13.105 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:27:13.108 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:27:13.120 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:27:13.122 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:27:13.123 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:27:13.123 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-09-10T18:27:13.124 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 +138us[ +185us] +/- 36ms 2024-09-10T18:27:13.124 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 20 +672us[ +718us] +/- 50ms 2024-09-10T18:27:13.124 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 -1030us[ -983us] +/- 64ms 2024-09-10T18:27:13.124 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:27:13.127 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:27:13.127 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-09-10T18:27:13.127 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 +128us[ +198us] +/- 36ms 2024-09-10T18:27:13.127 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 20 +675us[ +745us] +/- 50ms 2024-09-10T18:27:13.127 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 -1036us[ -966us] +/- 64ms 2024-09-10T18:27:13.128 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:27:13.128 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T18:27:13.136 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T18:27:13.136 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T18:27:13.144 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T18:27:13.234 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T18:27:13.301 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:27:13.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:27:13.329 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:27:13.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:27:13.360 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T18:27:13.369 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T18:27:13.377 INFO:teuthology.task.internal:Duration was 1347.712281 seconds 2024-09-10T18:27:13.377 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T18:27:13.385 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T18:27:13.385 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T18:27:13.387 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T18:27:13.423 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:27:13.439 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:27:13.800 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T18:27:13.800 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T18:27:13.800 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T18:27:13.861 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T18:27:13.861 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T18:27:13.885 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T18:27:13.886 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T18:27:13.903 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T18:27:13.952 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T18:27:13.953 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T18:27:13.969 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T18:27:14.240 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T18:27:14.250 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T18:27:14.250 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T18:27:14.276 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T18:27:14.301 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T18:27:14.311 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:27:14.318 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:27:14.339 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-09-10T18:27:14.364 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-09-10T18:27:14.385 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:27:14.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:27:14.406 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:27:14.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:27:14.440 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T18:27:14.449 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T18:27:14.450 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900198/remote/smithi099 2024-09-10T18:27:14.450 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T18:27:14.490 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900198/remote/smithi133 2024-09-10T18:27:14.490 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T18:27:14.525 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T18:27:14.526 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T18:27:14.528 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T18:27:14.575 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T18:27:14.586 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T18:27:14.586 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T18:27:14.597 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T18:27:14.597 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T18:27:14.599 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T18:27:14.612 INFO:teuthology.orchestra.run.smithi099.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 18:27 /home/ubuntu/cephtest 2024-09-10T18:27:14.630 INFO:teuthology.orchestra.run.smithi133.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 18:27 /home/ubuntu/cephtest 2024-09-10T18:27:14.631 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T18:27:14.640 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T18:27:14.662 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} duration: 1347.7122809886932 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-10T18:27:14.663 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T18:27:14.725 INFO:teuthology.run:pass